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STABILITY OF SADDLE POINTS VIA EXPLICIT 
CODERIVATIVES OF POINTWISE SUBDIFFERENTIALS 
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Abstract We derive stability criteria for saddle points of a class of nonsmooth optimiza¬ 
tion problems in Hilbert spaces arising in PDE-constrained optimization, using metric 
regularity of infinite-dimensional set-valued mappings. A main ingredient is an explicit 
pointwise characterization of the regular coderivative of the subdifferential of convex inte¬ 
gral functionals. This is applied to several stability properties for parameter identification 
problems for an elliptic partial differential equation with non-differentiable data fitting 
terms. 


1 INTRODUCTION 


This work is concerned with optimization problems of the form 
(1.1) min G{u) + F{K{u)) 

ueX 


for proper, convex and lower semicontinuous functionals G : X —> IR := [RU{oo} andf : T —> IR 
and a Frechet-differentiable operator K : X ^ Y between two Hilbert spaces X and Y, in 
particular for integral functionals on L^(Q) of the form F{u) - j^f{u{x))dx for a convex 
integrand / : IR —> [R. Under suitable regularity assumptions, a minimizer u e X satisfies 


I -p € VKiuTidFiKiu))), 

I p € dG{u), 


for somep € X*. Here, dF denotes the convex subdifferential of F and VK{u)* the adjoint of the 
Frechet derivative of K at u. Using convex duality, one can characterize the primal-dual pair 
{u,p) via the saddle point {u, v) of the Lagrangian 


(1-3) 


L{u, v) := G(u) + (K(u), v) - F*{v), 


where F* : 7 * —> R denotes the Fenchel conjugate of F; in this case, p and v are related via 
p = VK{u)*v. 
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To fix ideas, a prototypical example is the fitting problem [9] 

(1.4) min \\S(u) - y^H^i + 

Here, G(u) = K{u) = S(u) — y^, where S maps u to the solution y of-Ay + uy = f for 

given /, and T(y) = ||y||r,i. This formulation is appropriate if the given data y'^ is corrupted by 
impulsive noise. Here, F*{v) = i{|t;(jc)|<i}(i^), where ia denotes the indicator function of the set A 
in the sense of convex analysis [23]. A second example is the Morozov (constrained) formulation 
of inverse problems appropriate for data subject to uniformly distributed noise [8], 

(1.5) min—11^11,2 s.t. |S(u)(x) - y (x)| < (5 a.e.inQ. 

Here, G and K are as before, while F{y) = i{|y(x)|<5}(y) £^nd hence F*{v) - ( 5 ||t;||j;,i. A similar 
problem arises in optimal control of partial differential equations with state constraints. 

As we show in Section 4.1, critical points of the saddle point problem (1.1) may be characterized 
concisely through the variational inclusion 

0 € Ro(u, v), 


where we define the set-valued mapping Rq : X xY ^ X xYhy 


Ro{u, v) 


(dG(u) + VK{u)*v\ 

( dF*iv)-Kiu) j- 


Our goal then is to study the stability of solutions to (1.1) resp. (1.2) through set-valued analysis 
of this mapping. The main tool is a form of Lipschitz continuity of R^^ known as the Aubin 
property (or pseudo-Lipschitz or Lipschitz-like property). This is also called the metric regularity 
of Rq; see, e.g., [14, 24, 36,37, 45]. In contrast, second-order stability analysis for PDE-constrained 
optimization problems is usually focused on the stability of the optimal values and of minimizers 
(as opposed to saddle-points) and is based on sufficient second-order conditions based on direc¬ 
tional derivatives, often in stronger topologies; nonsmoothness typically arises from pointwise 
constraints or, more recently, sparsity penalties. We refer to [5, 6] as well as the literature cited 
therein. 

Since the problem (1.1) is nonsmooth, the first-order conditions involve proper sub differential 
inclusions and hence the second-order analysis required for showing metric regularity involves 
set-valued derivatives. Considerable effort has been expended on obtaining explicit representa¬ 
tions of these derivative, although up to now primarily in the finite-dimensional setting, e.g., 
in [18, 20, 21, 39, 40], with a focus on normal cones arising from inequality constraints. The 
difficulty in the infinite-dimensional setting stems from the fact that there exists a variety of 
more or less abstract definitions of such objects, see, e.g., [37], although more explicit character¬ 
izations can be obtained in some concrete situations [22]. Here, by exploiting the fact that the 
nonsmooth functionals are defined pointwise via convex integrands, we are able to explicitly 
compute regular coderivatives pointwise using the finite-dimensional theory from [45]; see 
also [37] for further developments on their calculus. One of the main contributions of this work 
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is therefore to further narrow the gap between the concrete finite-dimensional and the abstract 
infinite-dimensional settings. 

Besides being of inherent interest, e.g., for showing stability of the solution of the inverse 
problem with respect to 5 , metric regularity is also relevant to convergence of optimization 
methods. In the context of the saddle point problem for the Lagrangian (1.3), it is required for the 
nonlinear primal-dual hybrid gradient method of [47]. More widely, through the equivalence 
[4] of the Aubin property to the recently eminent Kurdyka-Lojasiewicz property [30, 41, 42], 
metric regularity is relevant to the convergence of a wide range of descent methods [1]. It can 
also be used to directly characterize the convergence of certain basic optimization methods [4, 
26, 27]. Metric regularity is also closely related to the concept of tilt-stability, mainly studied in 
finite dimensions, see, e.g., [15, 16, 32, 34, 43], but recently also in infinite dimensions [38, 40]. 
An extended concept incorporating tilt stability is that of full stability [31, 33]. 

We also note that when the non-linear saddle-point problem can be written as the minimiza¬ 
tion of a difference of convex functions - as any objective can [46] - detailed characterizations 
exist in the finite-dimensional case of local minima [48] and sensitivity [47]. Moreover, a set¬ 
valued analysis of the solvability of such programs with further symmetric cone structure 
is performed in [49]. In certain cases, with a finite-dimensional control u in an otherwise 
infinite-dimensional problem, it is also possible to do away with the regularizer G [12]. 

This work is organised as follows. In Section 2, we derive pointwise characterizations of 
second-order subdifferentials or generalized Hessians of integral functionals on and give ex¬ 
amples for several functionals commonly occurring in variational methods for inverse problems, 
image processing, and PDE-constrained optimization. These results are used in Section 3 to give 
an explicit form of the Mordukhovich criterion for set-valued mappings in Hilbert spaces, in 
particular for those arising from subdifferentials of the integral functionals considered in the 
preceding section. Section 4 further specializes this to the case of set-valued mappings arising 
from the first-order optimality conditions (1.2) and gives sufficient conditions for several stability 
properties such as stability with respect to perturbation of the data. Finally, Section 5 discusses 
the satisfiability of these conditions in the specific case of the model parameter identification 
problems (1.4) and (1.5), where it will turn out that stability can only be guaranteed after either 
introducing a Moreau-Yosida regularization or a projection to a finite-dimensional subspace 
in F. 


2 DERIVATIVES AND CODERIVATIVES IN L^iQ) 

Sadly, we cannot as in [50] directly use the clean finite-dimensional theory from [45] to show 
the Aubin property of R~^ through the Mordukhovich criterion [35]. We have to delve into the 
various complications of the infinite-dimensional setting as presented in [37]. The first one is the 
multitude of different definitions of set-valued generalized derivatives. Luckily however, as it will 
turn out, because of the pointwise nature of the non-smooth functionals whose second derivatives 
we require, we will be able to compute the pointwise differentials using the finite-dimensional 
theory, and limit ourselves to the regular coderivative in infinite dimensions. Although the 
results of this section hold for integral functionals on 1 / for any 1 < p < cxj, we restrict the 
presentation to the Hilbert space for simplicity (and since we will make use of the Hilbert 
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space structure of the saddle-point problem (1.2) later anyway). 


2.1 SET-VALUED MAPPINGS AND CODERIVATIVES 

We first collect some notations and definitions for set-valued mappings in Hilbert spaces, 
following [37,45] and simplifying the setting of the latter to Hilbert spaces. The symbols X, Y, Q, 
and W generally stand for (infinite-dimensional) Hilbert spaces, which we identify throughout 
with their duals via the Riesz isomorphism. With x € X, we then denote by B{x, r) the open ball 
of radius r > 0 . The closure of a set A we denote by cl A. 

Definition 2.1. Let U c X for X a Hilbert space. Then we define the set of Frechet (or regular) 
normals to L/ at u e f/ by 


N{u-,U) := {zeX 
and the set of tangent vectors by 


limsup -“Ao 

Usu'^u \W - m || 


(2.1) T(u; U):= {z€X 


exist r' \, 0 and €. U such that z = lim 


u — u 


I —»oo 


For a convex set U, these coincide with the usual normal and tangent cones of convex analysis. 

For our general results, we will need to impose some geometric regularity assumptions. 

Definition 2.2. We say that a tangent vector z e T{u; U) is derivable if there exists an e > 0 and 
a curve ^ : [0, e] —> L/ such that 


^+(0) := lim 

t\o 


m - m 


exists with ^.(.( 0 ) = z and ^( 0 ) = u. We say that U is geometrically derivable if for every u e U, 
every z € T{u; U) is derivable. 

It is easy to see that (cf [45, Prop. 6.2]) U is geometrically derivable if and only if T{u; U) 
for each u e U is defined by a full limit, i.e., we replace in (2.1) the existence of r' \ 0 by the 
requirement of the existence of u' for any sequence \ 0. 

For any cone V c X, we also define the polar cone 


(2.2) 


V° := {z € X I {z,z') < 0 for all z' eV}. 


We use the notation R : Q ^ W to denote a set-valued mapping R from Q to W; i.e., for every 
q ^ Q holds R{q) c W. For R: Q ^W, we define the domain domi? := {q €. Q \ R(q) + 0 } and 
graph Graph R := {{q, w) c Q X W | w € R{q)}. The regular coderivatives of such maps are 
defined graphically with the help of the normal cones. 
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Definition 2.3. Let Q and W be Hilbert spaces, and R : Q ^ W with domi? 0 . We then define 
the regular coderivative D*R{q\w) :W ^ Q of R at q e Q for w € W as 

(2.3) D*R{q\w){Aw) := jAq' e Q \ (Ag,-Aw) e N{{q,w); GraphR)^ . 

We also define the graphical derivative DR{q\'w) : Q W by 

DR{q\w){Aq) := {Aw € W \ (Aq, Aw) € T{{q,w); GraphR)} . 

The graphical derivative may also be written as [37, 45] 

/ , / , , R(q + tAq') - w 

(2.4) DR{q\w){Aq) = limsup -. 

f\0, Aq'^Aq ^ 

Here limsup^, Af stands for the outer limit of a sequence of sets {At c W}teT over an index 
set T, defined as 

limsupA^ := {w € W I for each i € l\l exist € T, w' e s.t. t' —> f and w‘ —> w}. 

t —»00 

Observe that DR{q\'w) : Q whereas D*R{q\w) :W^Q. Indeed, if R{q) = Aq for a linear 
operator A between Hilbert spaces, then for w = Aq holds 

DR(q\'w) - A and D*R(q\'w) - A*. 

The former is immediate from (2.4) (see also, e.g., [45, Ex. 8.34]), while the latter is contained in 
[37, Cor. 1.39]. 

We say that Graph J? is locally closed at {q, w), if there exists a closed set t/ c Q X W such that 
Graphic GU is closed. For any convex lower semicontinuous function / ; Q —> [R, the graph 
of the subdifferential df, considered as a set-valued mapping, is closed. This is an immediate 
consequence of the definition of the convex subdifferential. Finally, R is called proto-dijferentiable 
if Graph!? is geometrically derivable. 

2.2 SECOND-ORDER DERIVATIVES OF POINTWISE FUNCTIONALS 

Let X = L^(Q; R"") for an open domain Q c R” and G : X —> R be given by 

(2.5) G(u) = / g{x, u{x)) dx 

Jo 

for some integrand g : Qx R™ —> (-00, cxj]. Here we assume that 

(i) g is normal, i.e., the epigraphical mapping x 1—> epi g{x, ■) c R"" X R is closed-valued and 
measurable, 

(ii) g is proper and convex, i.e., the mapping z 1—> g{x, z) is proper and convex for each fixed 
X e Q. 
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(iii) dg is pointwise a. e. proto-dijferentiable, i.e., the mapping z i—> dzg{x, z) is proto-differentiable 
for a. e. X € Q. 

We call an integrand satisfying (i-iii) regular. Note that (i) already implies that the mapping 
z I—> g{x, z) is lower semicontinuous for each fixed x e Q and that g{x, u(x)) is measurable for 
each u e X [45, Prop. 14.28]. Examples of normal integrands are finite-valued Caratheodory 
functions [45, Ex. 14.29] and indicator functions of a closed-valued Borel-measurable mapping 
C : Q [R"" [45, Ex. 14.32]. Eor a normal integrand, (2.5) is well-defined, and G(u) < 00 if and 
only if u(x) e dom^(x, •) almost everywhere [45, Prop. 14.58]. 

Proto-differentiability is more restrictive but holds for a large class of practically relevant 
examples. In particular, under the present assumptions that g(x, •) is a proper, convex, lower 
semicontinuous function on a finite-dimensional domain, (iii) is equivalent to g(x, •) being 
twice epi-dijferentiable; see [45, Def 13.6] for the (technical) definition as well as [45, Prop. 8.41, 
Ex. 13.30, Thm. 13.40]. It is therefore satisfied, e.g., for the maximum of a finite number of 
functions [45, Ex. 13.16] and for proper, convex and piecewise linear-quadratic functions [45, 
Prop. 13.9]. More general ways to verify the proto-differentiability of dg{x, •) even without 
convexity include the concepts of full amenability [45, Def 10.23 & Cor- i 3 - 4 i] and prox-regularity 
[45, Def 13.27 & Thm. 13.40]. 

Since X - L^(Q; IR™) is decomposable, it suffices to have existence of at least one Uq € dom G 
to be able to compute pointwise the Eenchel conjugate 

G*{u) = f g*{x,u{x))dx 

Jq 

and the convex subdifferential 

(2.6) dG{u) = {^ e X I ^(x) e dg{x, u{x)) for a. e. x € Q}, 

where conjugate and subdifferential of g{x, z) are understood as taken with respect to z for x 
fixed; see [44, Thm. 3C] and [44, Cor. 3P], respectively. 

In order to calculate D*dG{x), we observe that 

Graph[ 5 G] = {(u, ^) € X x X | ^(x) € dg{x, u(x)) for a. e. x € Q}. 

Since g is normal and convex. Graph is measurable and closed-valued [44, Prop. 14.56]. Thus, 
in the definition of N{u, Graph[ 5 G]), we are dealing with a sequence in 

X X X = [L\Q-, R '”)]2 ^ R^"*). 

To derive an expression for D*dG, it therefore suffices to prove the following result. 

Proposition 2.1. LetU c L^(Q; R"") have the form 

L/ = {u € L^(Q; R™) | u(x) € C(x) for a.e.x€ Q} 

for a Borel-measurable mapping C : Q R'” with C(x) c R'” geometrically derivable for almost 
every X € Q. Then for every u e U we have 

(2.7) N(u-, t/) = |z e i^(G; R"*) | z(x) e N(u(x); C(x)) for a.e.xe q| , 
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and 


(2.8) 


T{u; U) = {z € L^(Q; IR™) | z(x) € T{u{x)-, C(x)) for a. e. x € Q} . 


Proof. We start with (2.7). Recalling the definition of N(u; U), we have to find all z € L^(Q; in'”) 
satisfying 

(z, n' - u) 

lim sup 


Usu'^u 11 ^ ^11 


< 0 , 


where the inner product and norm are now in L^{Q; R™), and the convergence is strong con¬ 
vergence in this space, within the subset U. 

Let us take a sequence —> u, {i = 1 , 2 , 3 ,...) and let e > 0 be arbitrary. We denote := u^-u. 

Then, 

(z,m' - u) f^{z{x),v'{x)) dx 


Li := 


\w - u\ 




1/2- 


We let Zi = Zj, Z2 and Z3 be sets with Lebesgue measure \ Zf) < e /3 for each j = 1 , 2,3 

and satisfying, respectively, the conditions 


(2-9) 

|r^‘(x)| < 3 e ^||t>'|| (x € Zj), 


(2.10) 

z is bounded on Z2, 


and 

/ n xl/2 


(2.11) 

/ \z{xtdx\ <e/ 3 . 



\jQ\Z-i 1 


To see how (2.9) can hold, we 

take Z/ as the set of x € Q satisfying (2. 

9). Then 

Al 

f \v\x)\^ dx > 3 £-^£'”{Q \ Zi)\\v‘\ 

||2 

1 9 


Jq.\Z‘ 


which gives 

£13 > £'”(Q \ Zi). 


We may therefore take Zj = Zj 

. The proofs that (2.10) and (2.11) can hold 

are similarly elementary. 

With 

z != z^ n Z2 n Z3, 


we have 

£"'(Q \ zf < £. 
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We calculate 


Li = 


dx ^ j^^{z{x),v\x)) dx 


lljQ\Z-z||||^il 
< -- + 


L 


{z{x),v'{x)) |t;‘(x)| 

’ —rr- dx 


|t;^(x)| 

II II , o -1 /" 

.. o -1 f {zix),v\x))\ 

Il3r.«.|ll|z|l+3z j 


dx. 
dx. 


< f^^^llzll + 3 e 


"L 


max < 0, 


(z(x),t;^(x)) ] 
t;‘(x)| J 


dx. 


If now for almost every x € Q we have that z(x) € Ar(u(x); C(x)), then we deduce using the 
boundedness of z on Z2 and reverse Patou’s inequality that 

limsupL; < f^^^||z||. 

i —^00 

Since f > 0 was arbitrary, we deduce 

N{u; f/) D {z e L^(Q; IR”’) | z(x) € N(u{x)\ C(x)) for a. e. x € Q}. 


This proves one direction of (2.7), which therefore holds even without geometric derivability. 
Now we have to prove the other direction, where we do need this assumption. 

So, let z € N{u; U). We have to show that z(x) € N(u(x); C(x)) for a. e. x € Q. Suppose this 
does not hold. Using the standard polarity relationship N(u(x);C(x)) = [r(u(x); C(x))]°, e.g., 
from [45, Thm. 6.28], we can find d > 0 and a Borel set £ c Q of finite positive Lebesgue 
measure such that for each x € £ there exists w(x) € T(u(x); C(x)) with ||w(x)|| = 1 and 
(z(x), w(x)) > S. We may without loss of generality take C(x) geometrically derivable for each 
X e £. By Definition 2.2 there then exists for each x e £ a curve ^(•,x) : [ 0 , e(x)] —> C(x) such 
that ^y( 0 ,x) = w(x) and ^( 0 ,x) = u(x). Let us pick c e ( 0 , 5 ). By replacing £ by a subset of 
positive measure, we may by Egorov’s theorem assume the existence of f > 0 such that 

|^(t, x) - ^(0,x) - w(x)t| < ct (t € [0, e], X € £). 


u\x) := 


Let us define 

[f(f,x), X € £, 
l^u(x), X € Q \ £. 

With := - u, we have v^(x) = ^(t,x) - ^( 0 ,x) for x e £, and i;^(x) = 0 for x e Q \ £. 

Therefore, for t e (0, e] and some c' > 0 there holds 




r r \i/2 

J |^(t,x) - ^(0,x)|^ dxj < ij (|w(x)|f + ct)^ dxj < c't. 


Likewise 


{z{x),v\x)) > (z(x), w(x))f - |z(x)| • |f(f,x) - ^(0,x) - wf| > St - ct. 










It follows 


lim sup 

f\o 


L 


(z(x),v^(x)} 


dx > lim sup 

t\o 


£"'{E){St - ct) 
c't 


£^{E ){5 - c) 
c' 


> 0 . 


With £ := for i e 1 ^, it follows that lim,^ooLi > 0 . This provides a contradiction to 
z € Ar(i<; U). Thus z(x) e N{u{xy, C(x)) for a. e. x e Q, finishing the proof of (2.7). 


We still have to show (2.8). The inclusion 


T(u; [/) c {z € L^(Q; IR™) | z(x) e T{u{x)\ C(x)) for a. e. x e Q} 

follows from the defining equation (2.1) and the fact that a sequence convergent in L^(Q) 
converges, after possibly passing to a subsequence, pointwise almost everywhere. 

For the other direction, we take for almost every x € Q a tangent vector z(x) e T{u{x)\ C(x)) 
at u{x) € C(x). For the inclusion (2.8), we only need to consider the case z € L^(Q; R'”). By 
geometric derivability, we may find for a. e. x € Q an e{x) > 0 and a curve ^{•,x) : [0, e(x)] —> 
C(x) such that ^( 0 , x) = u{x) and ^+( 0 , x) = z(x). In particular, for any given c > 0 , we may find 
Ec{x) € (0,£(x)] such that 


(2.12) 


|f(f,x)-^(0,x)-z(x)t| 


< c (f € (0, fc(^)]^ a. e. X e Q). 


For t > 0 , let us set 
If we define 

-c,t 


Ecj := {x e Q I £c(x) > t}. 


u‘^'\x) := 


[^(t,x), X e Ec^t, 
| m ( x ), X€Q\Ecj, 


then by (2.12), |u‘^’^(x) - m(x)| < t{c + |z(x)|) for a. e. x € Q, so that 


(2-13) 


- u\\ < ly t\c + \z{x)\f^ ' < t{cyj£'”{Q) 


+ z ). 


Moreover, (2.12) also gives 

(2.14) — - ^——<\if \^{t,x)-^{ 0 ,x)-z{x)t\^dx\ +;(/' \z{x)t\^dx 

< c4££^ + \\zxq\e,,M- 

For each i € l\l we can find ti > 0 such that \\zxQ.\Eyi II ^ 1 /f This follows from Lebesgue’s 
dominated convergence theorem and the fact that £"'{ 0 . \Ec^t) ^ ^ t ^ 0- The estimates 
(2.13) and (2.14) thus show that := satisfy u' ^ u and {£ — u)lti —> z. Therefore 

z € T{u; U), finishing the proof of (2.8). □ 

As a corollary, we may calculate D*dG{u\w) for G of the form (2.5). 
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Corollary 2.2. LetG : L^(Q; IR'”) —> R have the form {2.^) for some regular integrand g. Then the 
regular coderivative ofdG at u for ^ in the direction where u, € L^(Q; R"*), is given by 


Au{x) e D*[dg{x, •)](w(x)||'(x))(A^(x)) 

for a.e. X € Q 


D*[dG]iu\^)iA^) = JAu e L\Q-,R"') 

Likewise, the graphical derivative at u for f in the directon Au is given by 
D[dG]{u\^){Au) - Ia^ e 1 ^( 0 ; R'”) 


Af(x) € D[dgix, ■)](w(x)|f(x))(Au(x)) 
for a.e. X e Q 

Proof. As we have already remarked in the beginning of the present Section 2.2, the sets 

C(x) := {(z, ^) € R"’ X R"* I f € dg{x, z)} 


are by [45, Prop. 8.41, Ex. 13.30, Thm. 13.40] geometrically derivable for regular integrands g. The 
present result therefore follows by direct application of Proposition 2.1 to the set U = Graph 5 G 
with dG given in (2.6). □ 

More generally, we have the following. 

Corollary 2.3. LetP : Q ~ L^{Q; R"’) ~ L^{Q; R^) have the form 

P{q) = {w € L^(Q; R™) | w(x) € p(x, q{x))for a. e. x € Q} 


for some Borel-measurable and pointwise a. e. proto-differentiable set-valued function p : QxR™ 

R^. Then the regular coderivative ofP atqforw in the direction Aw, where q € L^(Q; R"^), and 
w, Aw e L^(Q; R*^), is given by 


D*P(q\w)(Aw) 


jAg € L2(Q;R'”) 


Aq(x) € D*[p(x, •)](g(x)|w(x))(Aw(x)), | 

for a.e. X e Q j 


Likewise, 


DP(q\w){Aq) - Aw € L^(Q; R™) 


Aw(x) e D[p(x, ■)](g(x)|w(x))(Ag(x)), | 
for a.e. X & Q f 


Corollary 2.4. LetP be a pointwise set-valued functional as in Corollary 2.3, and leth : Q ^ W 
be single-valued and Frechet differentiable. Then 

(2.15) D*{P + h){q\w){Aw) - D*P(q\w - h{q)){Aw) + ['Vh(q)]*Aw, 

and 


(2.16) D(P + h){q\w){Aq) - DP{q\w - h{q)){Aq) + Vh{q)Aq. 


Proof. Similarly to the finite-dimensional case in [45, Ex. 10.43], the rule (2.16) follows immedi¬ 
ately from the defining equation (2.4). The rule (2.15) is immediate consequence of the sum rule 
for regular coderivatives [37, Thm. 1.62]. □ 
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Remark 2.5. Using (2.7), it is not dijficult to obtain the characterization 

(2.17) N{u; U) - [z e L^{Q; [R'”) | z(x) e N(u{x)-, C(x)) for a.c.xe Q} 

of the limiting normal cone N{u; U) := lim N{u'; U). The proof is based on 1 ? conver¬ 

gence giving pointwise a. e. convergence for a subsequence, and in the other direction, reindexing 
finite-dimensional sequences to get sequences convergent in L?. The expression (2.17) then allows 
obtaining corresponding versions of the corollaries above for the limiting coderivative D* (which 
enjoys a richer calculus) in place of the regular coderivative D*. 

However, the stability analysis which is the focus of this work rests on the relation between the 
regular coderivative and the graphical derivative, discussed in the following section, which does 
not hold for the limiting coderivative (which has a similar relation with the regular derivative). In 
particular, we cannot work in the same way with the convexified graphical derivative which is a 
key step in our analysis (see Section 3.2 below). Hence, we do not treat this case in detail here. 

2.3 THE FINITE-DIMENSIONAL CODERIVATIVE IN TERMS OF THE GRAPHICAL DERIVATIVE 

Corollary 2.2 and Corollary 2.3 give us computable expressions for the coderivative for pointwise 
set-valued mappings in infinite dimensions in terms of the coderivative in finite dimensions. It 
is, however, often easier to work with the graphical derivative (2.4). From [45, Prop. 8.37] we 
find for P : R'” IR" that 

(2.18) D*R(q\w) = [DR(q'|w)]*'^. 

Here, for a general set-valued mapping J : Q ^ W, the upper adjoint J** : VC Q is defined 
via 

J*'''(Aw) := {Aq € Q \ {Aq, Aq) < {Aw, Aw') when Aw' € J{Aq')}. 

In general, the graph of the regular coderivative need not be a convex set. It is often more 
convenient - and for our analysis sufficient - to work with its convexification. To see this, 
observe first that by definition of the upper adjoint, and minding the negative sign of Aw' in 
(2.3), the relation (2.18) is equivalent to 

AI((q, w); Graph R) = r((q, w); Graph R)°. 

In particular - simply through the definitions of polarity and convexity, - the convex hull of 
the tangent cone satisfies 

AI((q, w); Graph R) = [convT((q, w); Graph R)]°. 

Defining DR{q\w) via 

GraphDR((3'|w) = convGraph[DR(q|w)], 

we therefore deduce 

(2.19) D*R(q\w) = [DR(q|w)]*+ - [DR(q|w)]*+, 

where the first equality holds due to the finite-dimensional setting, while the second equality 
holds generally due to the properties of convex hulls and polars. 

The following central results shows that for the pointwise functionals that are the focus of 
this work, both equalities hold even in the infinite-dimensional setting. 
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Theorem 2.6. LetP : Q ~ L^(Q; in'”) ~ L^(Q; n^) have the form 

P{q) = {av e L^(Q; in'”) | w{x) € p{x, q{x))for a.e.xe Q}. 

with proto-differentiable p{x, ■) : in'" U.^ (a.e.x €. Q) having locally closed graph at {q(x),w{x)) 
for a. e. X € Q. Then 

D*P{q\w) = [DP{q\w)Y* - [5P(<7|w)]*+, 
i.e., 

Aq e D*P{q\w){Aw) {Aq,Aq') < (Aw, Aw') when Aw' € DP(g|w)(Aq'') 

{Aq,Aq') < {Aw, Aw') when Aw' € DP{q\w){Aq'). 

Proof Let us define px ■= p{x, •). From Corollary 2.3, we have the equivalence 

Aw(x) e Dpx{q{x)\w{x)){Aq{x)) for a. e. x e Q Aw e DP{q\w){Aq). 

Provided that Graphp;^ is locally closed for a. e. x e Q, we may thus calculate 

(2.20) Aq e D* P{q\w){Aw) Aq{x) e D*px{q(x)\w(x))(Aw{x)) fora. e. xeQ 

(Aq(x), Aq'(x)) < (Aw(x), Aw'(x)) for a. e. x e Q 
when Aw'(x) € Dpx(q(x)\w(x))(Aq'(x)) for a. e. x e Q 
(Aq(x), Aq'(x)) < (Aw(x), Aw'(x)) for a. e. x e Q 
when Aw' € DP{q\w){Aq'). 

Here we are still computing the upper adjoint pointwise. Clearly (2.20) however implies 

(2.21) Aq e D*P{q\w){Aw) (Aq, Aq') < (Aw, Aw') when Aw'€ DP(q|w)(Aq'). 
Further, if there exists a set £ c Q with X'”(G \ £) > 0 and 

(Aq(x), Aq'(x)) > (Aw(x), Aw'(x)) (x e £), 

then by constructing 

Aq"(x) := (1 + fX£(x))Aq(x), Aw"(x) := (1 + t/£(x))Aw(x), 
we observe for sufficient large t the condition 

(Aq, Aq") > (Aw, Aw"). 

Moreover, by the pointwise character of P, also Aw" e DP(q|w)(Aq"). Thus the implication in 

(2.21) actually holds both ways, which is exactly what we set out to prove. 

Finally, [DP(q|w)]*''‘ = [DP(q|w)]*''‘ always holds, as we have already observed in (2.19). □ 

Similarly to Corollary 2.4, we have the following immediate corollary. 
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Corollary i.j. Let P be apointwise set-valued functional as in Theorem 2.6, and let h : Q ^ W be 
single-valued and Frechet differentiable. Then 

(2.22) D*(P + h)iq\w) = [D{P + /i)(g|w)]*+. 

Proof Let us set R '.= P -t-h, and recall from Corollary 2.4 that 

/S.q e D*R(q\'w)(Aw) = D*P{q\'w — h{q))(Aw) + [Wh{q)]*Aw. 

This is the same to say as that Aq := Aq - [Vh{q)]*Aw € D*P{q\w'){Aw) for w' := w — h{q). By 
Theorem 2.6 this holds if and only if 

{Aq,Aq') < {Aw, Aw') when Aw' € DP{q\w'){Aq'), 


or equivalently 


{Aq, Aq') < {Aw, Aw' + Vh{q)Aq') when Aw' e DP{q\w'){Aq'), 
which is just the same as 

{Aq, Aq') < {Aw, Aw') when Aw' € DP{q\w'){Aq') + Vh{q)Aq'. 

Now we just use (2.16) to derive (2.22). □ 

Corollary 2.8. LetG : L^(Q; IR'”) —> [R have the form {2.^) for some regular integrand g. Then the 
graphical derivative ofdG at u for ^ in the direction Au, where u, Au e L^{Q; R'"), is given by 


(2.23) D[dG]{u\^){Au) 




A^(x) € D[dg(x, ■)](u(x)l^(x)XAu(x)) | 
for a.e. X e Q f 


Moreover, 

D*{dG]{u\^) = [D[dG]{u\^)r. 


Proof The claim follows directly from Theorem 2.6 withp(x, z) = dg{x, z). Local closedness of 
Graph dg{x, ■) is a consequence of the lower semicontinuity of g. □ 


2.4 EXAMPLES 

We now study specific cases of the finite- and infinite-dimensional second-order generalized 
derivatives, relevant to our model problems (1.4) and (1.5). Other examples satisfying the as¬ 
sumptions are the piecewise linear-quadratic “multi-bang” and switching penalties introduced 
in [10] and [11], respectively. 
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2.4-1 SQUARED 1,2(0;^'") NORM 

The following result is standard; see, e.g., [45, Ex. 8.34]. 

Lemma 2.9. Withz € [R"", letg{z) - |||z||^. Then dg isproto-dijferentiable with 


D{dg){z\^){Az) 


j Az, ^ - z, 

I0, otherwise. 


From Corollary 2.8, we immediately obtain 
Corollary 2.10. With g(z) = |||z||^ and Q c IR” an open bounded domain, let 

G(u) := j g{u{x)) dx (u € L^(Q; IR'”)). 

Ja. 

Then 

D(dG)(u\^)(Au) - Au and D*{dG){u\^){A^) - A^. 


2.4.2 INDICATOR FUNCTION 

The following lemma is useful for computing D[dF*]{v\g) for the problem (1.4). Its claim in the 
one-dimensional case (m = 1 ) is illustrated in Figure 1. 

Lemma 2.11. Withz e IR™, let f{z) - iB{o,a){z)- Then df is proto-differentiable with 


(2.24) 


D{df){z\ 0 {Az) 


ll^ll Az/a + [Rz, 
[0, oo)z, 

■ { 0 }, 

{ 0 }, 


||z|| = a, ^ e (0,oo)z, (z, Az) = 0, 
||z|| = a, ll^ll = 0 , (z, Az) ^ 0 , 

||z|| = a, ll^ll = 0 , (z, Az) < 0 , 

l|z|| < a, ll^ll = 0, 

otherwise. 


In particular, ifm = 1 , then 


(2-25) 


D{df){z\ 0 {Az) 


IR, 

|z| = a, ^ € (0, cx3)z, Az = 0, 

[0, oo)z, 

|z| = a, ^ = 0 , Az = 0 , 

{0}, 

|z| = a, ^ = 0 , zAz < 0 , 

{0}, 

o' 

II 

V 

0, 

otherwise, 


as well as 


(2.26) 


D{df){z\ 0 {Az) 


IR, |z| - a, ^ e (0, oo)z, Az = 0, 

[ 0 ,oo)z, |z| = a, ^ = 0 , zAz < 0 , 

■ 

{0}, |z| < a, ^ = 0, 

0, otherwise. 
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(a)D(5/)(z|0 (b)Z^)(z|0 


ic) D*{df)iz \0 


Figure i: Illustration of the graphical derivative and regular coderivative for df with / = i[-i,i]- 
The dashed line is Graph df. The dots indicate the base points (z, where the graphical 
derivative or coderivative is calculated, and the thick arrows and gray areas the direc¬ 
tions of (Az, A^) relative to the base point. The labels (i) etc. indicate the corresponding 
case of (2.25). 


Proof. The proto-differentiability of df follows from the fact that / is twice epi-differentiable; 
see [45, Ex. 13.17 & Thm. 13.40], writing B{ 0 ,a) = {x e [R" | ||x||^ e (-00, a]} for the twice 
continuously differentiable mapping x 1—> ||x||^ and the polyhedral set (-00, a] satisfying the 
contraint qualification. 

For the full proof of (2.24), using second-order subgradient theory from [45], we refer to [50].^ 
For completeness, we provide here an elementary proof of the one-dimensional case (2.25). We 
have 

1 [0, oo)z, |z| = oc, 

{0}, |z| < a, 

0, otherwise. 

If ^ € df{z) and A^ € D{df){z\^){Az), there exists by (2.4) sequences P \ 0 , Az' —> Az, and 
€ df(z + t'Az') such that 

(2.28) A^ = hm - 0- 

i—>00 t 

We proceed by case distinction. 

(i) If |z| = a, Az = 0 , and ^ e ( 0 , oo)z, choosing z' = 0 , we can for any Af e IR and large 
enough i take = PA^ -I- ^ € [ 0 , oo)z = df{z). Thus we obtain the first case in (2.25). 

^There is a small omission in [50, Lemma 4.2], that actually causes D(df)(z\^)(Az) instead to be calculated. In 
calculating the subdilferential of (4.18) therein, at the end of the proof of the lemma, the cases (y, w) = 0 and 
{y, w) < 0 need to be calculated separately to give the two different sub-cases of (||z|| = a and ||f || = 0) in our 
expression (2.24). 
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(ii) Let us then suppose \z\ = a, Az = 0 , but ^ = 0 . In this case, choosing z' = 0 , we have 

by (2.27) free choice of e [ 0 , oo)z. Picking e [0, oo)z and setting := t'A^, we 

deduce that € [ 0 , oo)z c D{df){z\^){Az). Since -( 0 , oo)z is clearly not approximable 
from [0, cx3)z, we obtain the second case of (2.25). 

(iii) If |z| = a and zAz > 0 , then df{z + t‘Az') - 0 for large i. Therefore it must hold that 
zAz < 0 . If Az 7^ 0, it follows that - 0 (for large i). Since ^ is fixed, the limit (2.28) 
does not exist unless ^ = 0, in which case also A^ = 0 . This is covered by the third case 
of (2.25). 

(iv) If |z| < a, then ^ - 0 , so we get the fourth case in (2.25). 

(v) If |z| = a and Az = 0 , but ^ e -( 0 , oo)z, we see that 


sign^i(r-D>^l^l, 


so the limit (2.28) cannot exist. Therefore the coderivative is empty. 

Likewise, we obtain the empty coderivative if |z| > a, since even df{z) is empty and f 
does not exist. Together, we obtain the final case in (2.25). 

Finally, regarding D{df ){z\^) with m = 1, we see that only the case |z| = a and ^ = 0 is spHt 
into two sub-cases in (2.25), yielding an altogether non-convex Graph[D(d/)(z|^)]. Taking the 
convexification of this set yields (2.26); cf Figure 1. □ 

Corollary 2.12. Let f*{z) := i\-a,a]{^) 



Then 


D[dF*]{v\rj){Av) - 


VdF-ivlrjy, Av e VdF*{^\h) rj e dF*{v), 
0, otherwise, 


and 


D*[dF*](v\r])(Ar]) = 


VdF'-i'^lilT’ ^ V3F‘i'o\i]) and t] e dF*{v), 

0, otherwise, 


for the cone 


VdF'{^\ri) = {z € L^(Q) I z{x)v{x) < 0 i/|t>(x)| = a and z{x)r]{x) > 0} 


and its polar 


VgF*(^\f]T = {v € L^(Q) I v(x)v(x) > 0 ifr](x) = 0 and v(x) = 0 if\v(x)\ < a}. 
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Proof. The claim about the graphical derivative follows from Corollary 2.8 and Lemma 2.11, using 
the fact that the indicator function of a closed convex set is normal. The regular coderivative 
formula follows from the more general Proposition A.i in the appendix. Here, in the derivation of 
the explicit form of the polar cone VsF*{v\r])°, we use the fact that D[dF*]{v\r]){Av) is non-empty 
if and only if 

(2.29) r]{x)v{x) - \t]{x)\ and |f(x)| < a (x e Q). □ 

Remark 2.13. If(v, rj) satisfy the strict complementarity condition |i;(x)| < a or \r]{x)\ > 0 for 
a.e.x e Q, the degenerate second and third case in (2.25) (corresponding to the gray areas in 
Figure 1) do not occur, and the cone simplifies to 

Vffffv\r]) := {z e L^(Q) | z(x) = 0 i/|i;(x)| = a, x €. Q}. 

Note that points x € Q where a degenerate case occurs are precisely those where there is no graphical 
regularity ofdf at(v(x), fix)). We refer to [45, Thm. 8.40] for the definition of this concept, which 
we do not require in the present work. 


2.4.3 C(Q;R'") NORM 


The following lemma is useful for computing D[dF*] for the problem (1.5). Its claim in the 
one-dimensional case (m = 1 ) is illustrated in Figure 2. 

Lemma 2.14. Withz e [R™, let f*{z) - \\z\\2. Then df* is proto-differentiable with 

z^0,^||z||=z, 


{r}^ 

(2.30) D(dr)(z\f)(Az) = \{fy, 


0 , 


z - 0 , Az 0 , ^||Az|| = Az, 
z ^ 0 , Az ^ 0 , ll^ll ^ 1 , 
z ^ 0 , Az ^ 0 , ll^ll < 1 , 
otherwise. 


In particular, ifm = 1 , then 


(2-31) 


as well as 


(2.32) 


DidffiizlOiAz) = 


D(df*)(z\ 0 (Az) = 


{0} z 0, ^ = signz, 

{ 0 }, z = 0 , Az € (0,00)^, 

(-00, 0 ]^, z ^ 0 , Az ^ 0 , 1^1 ^ 1 , 

IR, z = 0 , Az = 0 , 1^1 < 1, 

0, otherwise. 


{0} z 0, ^ = signz, 

(-00,0]^, z ^ 0 , Az € [ 0 ,oo)^, 1^1 ^ 1 , 
R, z = 0 , Az = 0 , 1^1 < 1, 

0, otherwise. 
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{a)D{dniz\0 {b)D{dniz\0 (c)D*{dn{z\0 


Figure 2: Illustration of the graphical derivative and regular coderivative for df* with f* - 
I • I. The dashed line is Graph df. The dots indicate the base points (z, where the 
graphical derivative or coderivative is calculated, and the thick arrows and gray areas 
the directions of (Az, A^) relative to the base point. The labels (i) etc. indicate the 
corresponding case of (2.31). 


Proof. In the case m = 1, the proto-differentiability of df* follows from the fact that f* is 
piecewise linear and hence twice epi-differentiable; see [45, Prop. 13.9 & Thm. 13.40]. For general 
m e l\l, we may use the twice epi-differentiability of /(x) = established in the proof of 

Lemma 2.11 and the conjugate relationship in [45, Thm. 13.21] together with [45, Thm. 13.40]. 

It remains to verify the expressions (2.3o)-(2.32). We have for any m e N that 

d/*(z)-|lw}’ 

[clB(0,1), z = 0. 


We again proceed by case distinction. 

(i) For z 0 necessarily therefore D{df*){z\^) - 0 unless ^ = 2:/||z||, which yields the last 
case. 


(ii) If z 7^^ 0 and ( = z/||z||, for any 

z^ = z -t tAz'/ll Az' 


with z' z and t \ 0 , we have also df*(z') = z'/||z'||. The first case in (2.30) now 
follows immediately from computing the outer limit 


lim sup 

t'\0,Az'—^Az 


t 


lim sup 

t'\0,Az'—^Az 



= V 



Az. 


(iii) If z = 0 , and Az 0 , then z' 0 and z'/||z'|| = Az'/||Az'||. Therefore 


lim sup 

?\0, Az'—»A2 


t 


lim sup 

^\0, Az'—»Az 


Az'/IIAz'll-g- 

t 
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will only have limits if ^ lies on the boundary of B{ 0 , 1 ), and indeed ^ = Az/|| Az||. This 
gives the limit i.e., the second case. 

(iv) If z = 0 and Az = 0 , then we may pick ^ € cl B{ 0 , 1 ) arbitrarily by choosing also Az' = 0 . 
If II ^11 = 1, then we obtain the limit 

hmsup J( 5 ( 0 ,l) - 0 - {A^ € I (A^,0 < 0} - {r}° 

t —^OO f 

and hence the third case. 

(v) In the same situation, choosing ||^|| < 1 gives the limit [R"* and hence the fourth case. 

Finally, (2.31) is a trivial specialization of (2.30), while regarding D{df*){z\^) with m = 1 , we 
see that only the case z = 0 and |^| = 1 is split into two sub-cases in (2.31). These produce an 
altogether non-convex Graph[D(d/*)(z|^)]. Taking the convexification of this set yields (2.32); 
cf Figure 2. □ 


Corollary 2.15. Let f*{z) := 8\z\ and 


Then 


F*{v) 



{v € L^CQ)). 


and 


D[dF*]{v\r]){Av) - 


iVaF*{v\r])°, 

1 ®’ 


Av e ^ dF*{v), 

otherwise, 


D*[dF*](v\rj)(Arj) 


iVaF*{v\r]y, -Ar] € VaF-{v\r]) andrj e dF*{v), 
10, otherwise, 


for the cone 

VaF-ivlr]) - {z e L^{Q) \ z{x)r]{x) > 0 ifv{x) - 0 and {5 - |7(x)|)z(x) = 0}, 
and its polar 

VaF*i'^\h)° - {v e L^(Q) | v{x)t](x) < 0 if\t]{x)\ - S and v{x)v{x) - 0}. 


Proof. The claim about the graphical derivative follows from Corollary 2.8 and Lemma 2.14, 
using the fact that g{z) - \z\ is finite-valued and Lipschitz continuous and hence normal. The 
regular coderivative formula follows from the more general Proposition A.i in the appendix. To 
derive the explicit form of the polar cone VaF*{'^\h)°, we employ the fact that D[dF*]{v\r]){Av) 
is non-empty if and only if 


(2.33) 


\r]{x)\ < S and v{x)r]{x) = |f(x)|. 


□ 
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Remark 2.16. If{v,r]) satisfy the strict complementarity condition v{x) 0 or \r]{x)\ < 5 for 
a.e.x € Q, the degenerate second and third case in (2.31) (corresponding to the gray areas in 
Figure 2) do not occur, and the cone simplifies to 

VdF*{‘^\f]) ■— ^ L^{Q) I z(x) = 0 ifv(x) = 0, X e Q}. 

Again, points x € Q where a degenerate case occurs are precisely those where graphical regularity 
fails to hold for df* at {v{x), r](x)). 

2.4.4 SPATIALLY VARYING INTEGRANDS 

Let a,f€ L^(Q) with a(x) < f{x) for a. e. x e Q. Define 

f{x,z) := I[a(xii 3 (x)\{z) (x € Q;z e [R). 

This example is useful for spatially or temporally varying “tube” constraints, which arise in the 
regularization of inverse problems subject to variable noise levels [13]. The indicator function 
of temporally variable constraints also appears in Moreau’s sweeping process, which is a model 
for several phenomena from nonsmooth mechanics such as elastoplasticity [29]. 

Due to the measurability of a and f, the integrand / is proper, convex and normal [45, 
Ex. 14.32], such that the subdifferential df{x, ■) can be computed pointwise. Furthermore, / is 
a. e. proto-differentiable as the indicator function of the convex polyhedral set [a(x), ^(x)]; see 
again [45, Ex. 13.17 & Thm. 13.40]. By simple pointwise application of Lemma 2.11 we can thus 
compute D[df{x, •)]. We therefore deduce the applicability of Corollary 2.8 to 

H'^) = f fix, ^(x)) dx, 

Jq 

and obtain a pointwise characterization of D(dF) similar to Corollary 2.12. 

Clearly, we can analogously modify Corollary 2.10 (squared L^(Q; R"’) norm) and Corol¬ 
lary 2.15 (L^(Q; R"’) norm) by, e.g., introducing a spatially varying weight in each norm. 

3 STABILITY OF VARIATIONAL INCLUSIONS 

To pave the way towards studying the stability of saddle point systems in the following section, 
we now recall general concepts for the study of variational inclusions and develop general 
results that quickly specialize to saddle point systems in I?. 

3.1 METRIC REGULARITY AND THE MORDUKHOVICH CRITERION 

Our stability analysis is based on the following set-valued Lipschitz property [2, 37, 45], also 
known as the Aubin property of 

Definition 3.1. We say that the set-valued mapping R : Q ^ W is metrically regular at w for q if 
Graph R is locally closed and there exist p, 5 ,£ > 0 such that 

(3.1) inf \\q -p\\ ^ (W'w - R{q)\\ for any q, w such that ||q - q|| < < 5 , ||w - wH < p. 
p: weR{p) 
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We denote the infimum over valid constants i by or £ji-i for short when there is no 

ambiguity about the point (w, 

A simplified view, indicating why this concept is useful, can be seen by taking q satisfying 
0 e R(^- Setting q = q and w = 0 in (3.1), we then obtain 


(3.2) inf 115 ^-pll ^ ^_R-i(0|§)ll''^ll for any w such that ||w|| < p. 

p: weR{p) 

Therefore, if we perturb the variational inclusion 0 e P( 5 ) - typically an optimality condition - 
by a small linear perturbation w, we will still find a nearby solution to the perturbed problem. 
We will later see that for our problems of interest, we can encode variations in data and in 
an additional Moreau-Yosida regularization parameter into w. We therefore need to estimate 
f’p-i, for which the following Mordukhovich criterion [35] will be useful. It is also contained in 
[37, Thm. 4.7] and simplified here to our Hilbert space setting from the original Asplund space 
setting. 

Theorem 3.1. LetR : Q ^ W be a set-valued mapping between Hilbert spaces Q and W. Suppose 
Graph P is locally closed around {q, w) e Graph P. Then 


iR{q\w) = inf sup 


\\D*R(q'\w')\\ q € B{q, t), w' € R{q) D B(w, t) 


Here, for positively homogeneous M : W ^ Q, we have defined 


||M|| := sup{||(3'|| I q e M{w), ||w|| < 1}. 


If R satisfies the regularity assumption D*R{q\w) - [DR{q\w)]** (which is the case for point- 
wise mappings due to Theorem 2.6), we may translate Theorem 3.1 to be expressed in terms of 
the graphical derivative DR, where by the second equation in (2.19) it suffices to consider the 
convexification DR. This is the content of the next proposition. 

Proposition 3.2. LetR :Q^W be a set-valued mapping between Hilbert spaces Q and W. Suppose 
Graph!? is locally closed around {q, w) e Graph!? and 


(3-3) 


D*R{q\w) = [DR(q\w)f^. 


Then 

w' € B(w, t), q' € B{q, t), w' £ !?(q 0 | > 

with 


£R-i{w\q) = inf sup 


4-i(w'|q') 


(3-4) 


4-1 (w'I q') := sup 



Aq e Q, Aw £ W, ||Aq|| < 1, satisfying 
{Aq,Aq') < (Aw, Aw') when Aw' £ DR{q'\w'){Aq') 


Proof. From the Definition 2.3 of D*!?(q|w) and D*R ^(w|g) through JV((w, 1); Graph!?), we 
observe from the definitions that 


Aw £ D*R(w\q)(Aq) —Aq £ D*R ^(q|w)(-Aw). 
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Applied to R Theorem 3.1 therefore gives 

(3-5) 4 -'(w|g) ^ inf sup |||D*i?“\w'|g')ll ^ B{m/, 0 . <?' ^ R~\w') n B{q, t) 

- inf sup I ||[D*]?((5''|w')]“^|| w' e B(w, t), q € B{q, t), w' € R(q') 


^>o 


= inf sup < II Aw 


^>o 


Aq € D*R(q'\'w')(A'w), ||Ag|| < 1 , 
w' € B{w, t), q' e B{q, t), w' € R(q') 


- inf sup |^jj-i(w'|(7') w' e B{w, t), q e B{q, t), w' € R{q')j, 

where 

£ji-i{'w'\q') := sup |||Aw|| Aq € D*R{q'\w'){A'w), ||Aq'|| < l|. 
Referral to (3.3) and the fact that 

[DR{q\w)r = [DR{q\M,)r, 

now establishes the claim with the expression (3.4) for £j^-i{w'\q'). 


3.2 GRAPHICAL DERIVATIVES EXPRESSED WITH LINEAR OPERATORS AND CONES 


We now derive necessary and sufficient conditions for the Aubin property to hold for variational 
inclusions involving second-order set-valued derivatives of pointwise functionals. As seen in 
Section 2.4, these commonly have the structure of a sum of a linear operator and a cone. In fact, 
for the following analysis, it suffices that the graphical derivatives merely contain such a sum 
in order to derive upper bounds; this will be important for treating discretization by projection 
in Section 5.3. We therefore assume therefore that W = Q = L^(Q; [R^) and that 


( 3 - 6 ) 


DR(q\w)(Aq) D 


iTqAq + V(q\wr, 

1 ®, 


Aq e V{q\w), 
Aq ^ V(q\w), 


for some linear operator T '■= Tq '■ Q ^ Q, dependent on q but not w, and a cone V := V{q\w) c 
Q, dependent on both q and w. Here we recall from (2.2) that V° is the polar cone of R. Although 
it will not be needed in our analysis, an explicit characterization of the regular coderivatives of 
set-valued mappings satisfying (3.6) (with equality) is derived in Appendix A for completeness. 

Following the reasoning in [50, Prop. 4.1], we may, using the structural assumption (3.6), 
continue from Proposition 3.2 to derive 





Aq € Q, Aw € Q, ||Aq|| < 1, satisfying 

(3-7) 

4-1 (w|q) < sup ■ 

IIAwll 

(Aw, TAq' + Ap') < {Aq, Aq') 

for Aq' € V, Ap' e W 


= sup 



Aq € Q, Aw € V, ||Aq|| < 1 , satisfying 1 
(Aw, TAq’) < {Aq, Aq') for Aq' eV ] 


= sup 



Aq € Q, Aw € V, ||Aq|| < 1 , satisfying 1 
(T*Aw - Aq, Aq') < 0 for Aq' € R J 


= sup {||Aw| 
= sup < II Aw 


Aq € Q, Aw e y, IIAqII < 1, T*Aw - Aq e V°} 
Aw € V, inf ||T*Aw - z|| < ll . 

zeV° 
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Figure 3: A geometric illustration of the distance iji-i computed in (3.7) for (3.6). The dashed 
line indicates the transformed cone T*V. Without loss of generality, we restrict Aw to 
he on the unit sphere (dotted), in which case the distance between points on the unit 
sphere between the polar V° and T*V gives the inverse Lipschitz constant. 


We illustrate this expression geometrically in Figure 3. Observe also that if (3.6) holds as an 
equality, then so does the first inequality in (3.7). That is, in this case 


4-1 (w I q) = sup 



Aw e y, inf ||T*Aw-z|| 

zeV° 


< 1 


Remark 3.3. IfV is a closed subspace, then V° = V-'-, and (3.7) reduces to 

< sup{||Aw|| I Aw € V, ||PvT*Aw|| < 1} 

= sup{||PvAw|| I Aw € Q, ||PvT*PvAw|| < 1} . 


We can use (3.7) and the expansions above to estimate ^^-i(w'|q') for R = Hu and R = Rq. To 
study stability and metric regularity, we however still need to pass to 


4 -i('^^l< 7 ) = inf sup l^jj-i(w'lq') w' e B{w,t), q € B{q,t), w' € R{q') 


This in essence involves a uniform c > 0 in the condition 


inf ||T*,Aw - z\\ > c||Aw|| (Aw € V(q'\-w')) 

zeV{q'\Mf')° ^ 

for all {q', w') close to (q, w). 

If we assume continuity of the mapping q' 1—> Tq/, we can simplify this condition. The 
following lemma prepares the way for the stability analysis of saddle points in the next section 
(cf. (4.9) below). 
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Lemma 3.4. Let q,w e Q = W = X X Y, and suppose that for {q', w') in a neighborhood U of 
{q, w). Graph fMJ is closed, (3.3) holds, and we have 


(3.8) 


DR{q'\w'){Aq) D 


iTq^Aq + V(q'\wr, 
1 ®’ 


Aq € V{q'\w'), 
Aq t V(q'\w'), 


for a cone V{q'\w') c Y. In addition to these structural assumptions, assume the continuity at q of 
q' I—> Tqi, and for some c > 0 the bound 


(3-9) 


\\T*Aw-z 

a{q\w;R) := sup inf - .. . .. 

f>0 (Aw,z)eWjJ(ij|w), lieu’ll 

Aw^^O 


> c, 


where 


W^{q\w) := {V{q'\w') X V{q'\w')° \ w' € R{q'), \\q' - g'|| < t, \\w' — ivH < f} c Y^. 

Then 


(3.10) < c . 

Moreover, if (3.8) holds as an equality, then Lj^-i{w\q) < 00 if and only ifa{q\w;R) > 0 . 

Proof Suppose (3.9) holds, and pick ci € ( 0 , c). Whenever t > 0 is small enough and w' and q' 
satisfy 

w' e R(q'), \\q — q'W < t and ||w - w'|| < t, 

the bound (3.9), the continuity of q' 1—> T^/, and the inclusion 

V{q'\w') X Viq'lw'f C W^{q\w), 

guarantee the estimate 


lir^/Aw - z|| > Ci||Aw|| (Aw € V{q'\w'), z e V(q'\w')°). 

The latter says that 

lR-fw'\q') < cf. 

By (3.7) and (3.5), therefore 

Lf;-i(w'\q') = sup{^j^-i(w'|g') | w' € R(q'), Hg - q'\\ < t, ||w - w'\\ < t} < cf. 

Since c\ e ( 0 , c) was arbitrary, this proves (3.10). 

If (3.9) does not hold, and (3.8) holds as an equality, we can, given c > 0 , find for every t > 0 
a pair (Aw, z) € W^{q\w) \ { 0 } X Y, such that 

||r*Aw - z|| < e||Aw||. 

Thus, by the definition oiW^{q\w), we can also find q' and w' satisfying 
w' e R(q'), llq - q'W < t and ||w - w'|| < t 
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such that 


Aw € V(q'\w') and z €. V(q'\w')°. 

Recalling (3.7), which holds as an equality under the present assumption that (3.8) holds as an 
equality, this implies that 

Since t > 0 was arbitrary, we have as well that 

4-i(w|q) > 

Finally, since e > 0 was arbitrary, it follows that (j^-i{'w\q) = 00 if (3.9) does not hold. □ 


4 STABILITY OF NON-LINEAR SADDLE POINT SYSTEMS 


We now apply the results of the preceding section to saddle points characterizing minimizers of 
nonsmooth optimization problems of the form (1.1). In particular, we assume that 


(4-1) 



f*{v{x))dx 


for a proper, convex, lower semicontinuous f* and, motivated by the problems considered in 
the next section. 


(4-2) 


G{u) = 


/ g{u' 
Jo. 


(x)) dx for 


/ N “11' 

g{z) = -|z|' 


4.1 NON-LINEAR SADDLE POINT SYSTEMS AS VARIATIONAL INCLUSIONS 

We first write the first-order optimality conditions (1.2) for the problem (1.1) as an inclusion for 
a set-valued mapping and compute its derivative. 

For q - {u,v) to be a saddle point of (1.3), the Lagrangian L has to satisfy 

L{u, v) < L(u, v) < L(u, v) (u € X, V € 7 ). 

Since —L(u, •) is convex, proper, and lower semicontinuous for any u € X, we deduce from the 
necessary and sufficient first-order optimality condition 0 € d(—L(u, for convex functions 
together with the sum rule [17, Prop. 5.6] that K{u) e dF*{v). We also see that 

u e argmin G{u) + {K{u),zi). 

U 

Since G is convex and K € C^(X; Y), we can apply the calculus of Clarke’s generalized derivative 
(which reduces to the Frechet derivative and convex subdifferential for differentiable and convex 
functions, respectively; see, e.g., [7, Chap. 2.3]) to deduce the overall system of critical point 
conditions 

I K{u)€dF*(d), 

\-[\/K(u)]*dedG(u). 
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This may be rewritten concisely as 

(44) 

for the monotone operator 


0 e 


( 4 . 5 ) c,~K{u)-vmu. 

This is defined at an arbitrary base point u € X for the linearization of K. Here and generally 
we use the notation 


q = {u,v) e X xY and w - {^,q) e X xY 

for combining (primal, dual) and (co-primal, co-dual) variable pairs, respectively. This nomen¬ 
clature stems from v being the dual variable in the original saddle-point problem, whereas the 
co-primal and co-dual variables generally satisfy w e 

Alternatively, we may rewrite the critical point conditions (4.3) as 

0 ^ 


for 

(4.6) 


Ro{u, v) := Huiu, v) 


(dG(u) + WK(u)*v\ 

( dF*iv)-Kiu) j- 


The mapping Rq will be useful for general stability analysis, while Hu is critical for the primal- 
dual algorithm of [50]. 

We can prove the following about these mappings. 

Proposition 4.1. Let G : X - L^(Q; [R"") —> (-00, cxj] and F* : Y - L^(Q; IR") —> (-00, cxj] have 
the form (2.5)/or some regular integrands g and /*, respectively. LetK € C/X; T). Then GraphHo 
is locally closed, and 


(47) 


DHu{q\w){Aq) ^ 


I D[dG]{u\^ - VK{u)*v){Au) + VK{u)*Av \ 
\D[dF*](v\q + VK(u)u + Cu)iAv) - VK(u)Auj ’ 


with D[dG] and D[dF*] given by (2.23). Moreover (3.3) holds, i.e., 

D*Huiq\w) = [DHuiq\w)r. 


Proof. That Graph Ha is locally closed is an immediate consequence of the lower semicontinuity 
of the convex functionals G and F* and the continuity of VX. The expression (4.7) is an immediate 
consequence of Corollary 2.7, where we set 




and observe that h is not only smooth but linear with 


Vh{u, v) = 


0 

-Y!K{u)u 


VK{u)* 

0 
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Proposition 4.2. Let G : X = L^{Q; [R'”) —> (-00,00] and F* : Y = L^{Q; IR”) —> (-00, cxj] have 
the form (2.5) for some regular integrands g and f*, respectively. LetK € C^{X; Y). Then Graph J?o 
is locally closed, and 


(4.8) 


DRo{q\w){Aq) = 


lD[dG]{u\^ - VK{u)*v){Au) + VK{u)*v]Au + VK{u)*Av 
\ D[dF*]{v\r] + K{u))(Av) - VK{u)Au 


with D[dG] and D[dF*] given by (2.23). Moreover (3.3) holds, i.e., 

D*Roiq\w) = [DRoiqHTf 


Proof Again, the fact that Graph Rq is locally closed is an immediate consequence of the lower 
semicontinuity of the convex functionals G and F* and the continuity of VK. The expression 
(4.8) is also again an immediate consequence of Corollary 2.7, where we set 


ho{u,v) 


IWK(u)*v\ 

\ -Kiu) j 


and P(u, v) 


jdG(u)\ 

\dF%v)l’ 


and observe that 


Vho{u,v) ^ 


where we denote := V(u 

differentiable. 


Vu[VK(u)*v] VKiuf 
-Y!K{u) 0 

[VfC(u)*t;])(u), using the assumption that K is twice 

□ 


Remark 4.3. Observe from (4.7) and (4.8) that ifu = u. 


DRo{q\w) = DHu{q\w) + 


^u[^K{u)*v]Au^ 


Comparing (4.5) and (4.6) also shows that in this caseRo{q) = Huiq). 

Recalling (3.1) and (3.2), as well as Proposition 3.2, we see that in order to analyze the stability 
of (1.2), resp. (4.4), we have to compute {^-i{w'\q') in a neighborhood of (q, 0 ). We will later see 
that this will be necessary both for u = u and u = u'. 


4.2 LIPSCHITZ ESTIMATES FOR SADDLE POINTS 

We now derive sufficient conditions for the Aubin property to hold for saddle points of (4.3). 
We proceed in several steps. First, we observe that provided that if both D[ 5 G] and D[dF*] have 
individually the form (3.6), then the convexified graphical derivative DHu{q\w){Aq) also has 
the form (3.6). More precisely 


(4-9) 




G, Kl 
-Ka F, 


for some linear operators Gq : X ^ X and : 7 — > Y and Kq = VK{u), as well as the cone 
V{q\w) = VdGiu\^ - Klv) X Vapfvlq + KuU + c^) c X X Y. 
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Since G is assumed to be quadratic, we have Vdciu]^') = X, which gives the more specific 
structure 


D[dG]{u\f){Au) - GqAu 

and 

= I'"’''” ''■’'•(''I'''*"' ''"'’■(''I"'*- 

I®, AviVaF^{v\rj'). 

We make, of course, the implicit assumption that e dG{u) and rj' e dF*{v)\ if this does not 
hold, then the respective graphical derivatives are empty. 

As we will see, it is difficult in general to guarantee the Aubin property. One way of doing so 
is to consider a Moreau-Yosida regularization of F, that is to replace F* by 

F*Jv) := F*iv) + ^v\\^ 

^ 2 

for some parameter y > 0 ; see, e.g., [3, Chap. 12.4]. The regular coderivative of the regularized 
subdifferential satisfies at least at non-degenerate points for some cone VaF*{'^\f]) the expression 


(4.10) 


D[dF*]{v\r]){Av) 


{yAv + V 3 F-{v\r}T, Av € VgF'ivlt]), 
[0, Av t VdF*i'^\ri)- 


We denote the corresponding operator by ffj, j. 

From Proposition 4.2, we observe that DRo{q\w){Aq) also has the form (3.6) with (4.9), albeit 
with a different term Kn and with Gq including the second-order term Vu[VFr(u)*i;] from K. 


We now specialize the results of Section 3 to the specific setting considered in this section. 
We therefore assume that Fq = yl for some y > 0 and that Vgc - X. For the statement of the 
next lemma, we drop many of the subscripts and denote for short T := Tq, K := Kq, G := Gq, 
and V := VgF*(‘^\>l)- 

Lemma 4.4. Let V = X xV <z X xY be a cone, and letG : X —> X and K : X ^ Y be bounded 
linear operators. For y > 0 , define 

IG K* 

[-K yij 

Suppose G is self-adjoint and positive definite, i.e., there exists cg > 0 such that 


T := 


(4.11) > 4 ii?ir 

Then, there exists c > 0 such that 


ifeX). 


(4.12) inf ||T*w - z||^ > c||w||^ (w € V) 

zeV° 

if and only if either of the following conditions hold: 
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(i) Y > 0, in Mihich case c = c(f, cg); 
(a) there exists ck,v > such that 


(4.13) inf II^G ^K*t] - v||^ > CK,v\\n\\^ ih e V), 
veV° 

in which casec - c(||i?||, cg, < ck^v- 

Proof. We first prove the sufficiency of (i) and (ii). With w = € Y = X xVgp*, and 

z = (0, v) ^V° = {0} X Vqj.., we calculate 

\\rw-zf = m-K*r^f + m + Yri-vf 

= mf + \\rrtf -2{G^,K*rj) 

+ \\m^ + r'll'fll' + Ikll' + 2y{K^, rj) - 2{K^, v) - 2 y(; 7 , v) 

= IIG^II^ + 11^,7112 - 2{G^,K*r,) + ^ - v||^ + Y^M^ + MH, h) - ^Yih, v). 

Assume first that y > 0. For arbitrary A e [0, y], we can insert the productive zero and use 
~ y)(V’ h) - ^ foi' all V € y° and r] € V to obtain 

lirw - z||2 = IIG^II^ + \\K*r]f - 2((G + A7 - Yl)^,K*rj) 

+ 2X{K^,r,) + m-vf + Y^\\r]\\^-2Y{r],v) 

> mf + \\K*,j\\^-2{iG + Al-Yl)^,K*,j) 

+ 2A{K^-v,rj) + \m-vf + Y^rjf. 

This we further estimate by application of Young’s inequality for any pi, P2 > 0 as 

(4.14) lirw - z||2 > ||G^||2 + \\K*rjf - 2((G + AI- Yl)^,K*rj) 

+ (y^-Apr')ll'7ll^ + (l-Api)||^^-v||2 
> ||G^||2 - p-i||(G + A7 - y7)^||2 + (1 - p2)\\K*rj\\^ 

+ iY^-Ap;^)M^ + il-Ap,)m-v\\\ 

Let us choose pi = A“^ and p2 = 1. Then (4.14) becomes 

lirw - z||^ > IIG^II^ - ||(G + A7 - y7)^||2 + (y2 - A2)||,7||^ 

Now, 

||G^||2 - ||(G + A7 - y7)^||2 = 2(y - A)(G^, ^) - (y - A)2||^||2, 
so that by (4.11) we therefore require that 

2(y - A)cg - (y - Af > 0. 

This holds if A < y is large enough, verifying case (i) including the relationship c = c(y, cg)- 
Suppose next that y = 0. To verify the sufficiency of (ii), we proceed by contradiction, assuming 
(4.12) not to hold for 

_ CK,V 

^ ~ 1+ii7?G-iir 
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Thus, for some c' e ( 0 , c), we can find w € X X V and v € satisfying 

||rw-(0,v)||^ < c'llwll^. 

We may assume that ||w|| = 1. Thus, 

T*w-( 0 ,v) = (61,62) where ||ei||^ + ||e2||^ < c', 

which means 

- K*r] - Cl and — v = 62- 
Since G is invertible by (4.11), this shows that 

KG-^K*r]-v ^ ez-KG-^ei. 

Thus 

\\KG-^K*t] - vf < 2(1 + \\KG-%c' < CK,v, 

in contradiction to (4.13). Therefore (ii) is sufficient for (4.12). We also estimate 

IIGfll >cg||^||. 

Using the standard relation 

ur-ipu 

sup ||G ^11 - sup 


we therefore have 


\\KG-^\ < c-i||X||. 


This verifies c = c(||X||, cg, ck,v)- 

Having dealt with the sufficient conditions, let us now verify the necessity of (4.13) when 
y = 0 . We expand 

(4.15) \\rw-zf = \\G^-K*rjf + m-vf. 

Using the invertibility of G from (4.11), let us choose ^ = G~^K*r]. Then (4.15) gives 

\\T*w-z\\^ - ||XG“^X>- v||^ 

immediately showing the necessity of (4.13) and ck,v ^ c. □ 

Remark 4.5. It is easily seen that ify = 0 , then existence of a c > 0 such that 

||r;7|| > cll^ll (rj^VarG 
is necessary for the satisfaction of (4.12). 

We now combine the above low-level lemma with Lemma 3.4. 
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Lemma 4.6. Let q,^ e Q = W - X X Y and suppose that for {q', w') in a neighborhood U of 
{q, w). Graph fMJ is closed, (3.3) holds, and we have 


(4.16) 


DR{q'\w'){Aq) D 


iTq,Aq + VR{q'\wr, 
1 ®’ 


Aq € VR{q'\w'), 
Aq ^ VR(q'\w'), 


for 


and 




(4-17) 


VR{q'\w') - X X V{v'\q') <z XxY. 


In addition to these structural assumptions, suppose that the mappings q' 1—> Gq' and u' 1—> Ku' 
are continuous at q and u, respectively. Assume, moreover, that each Gq/ is self-adjoint and positive 
definite, i.e., there exists cg > 0 such that 


(4.18) 


{Gq^,^)>CGMf i^eX). 


Define further 


(4-19) 


Then 


b{q\w,R) := sup inf 

t >0 ((0,Aj;),(0,v))6W^((j|w), 


WKuGfK^Aq - V 

IIA/7II 


(4.20) LR-i{w\q) < 00 
provided 

(4.21) ma.x{Y,b(q\w;R)} > 0. 


If (4.16) holds as an equality, then (4.20) holds if and only if (4.21) holds. 

Proof. If p > 0 , we may directly apply Lemma 3.4. So we take y = 0 . Suppose first that (4.21) 
holds. Then b{q\w;R) =: ck,v > 0 , and (4.19) gives 


(4.22) 


WKuG-fKlAq - V 

IIA/7II 


> Ck,V 


for every Aq Q and v satisfying 


(( 0 ,Aq),( 0 ,v)) e W^(qlw). 

That is, using the facts that 0 € X and 0 € X°, as well as the expression (4.17), we see that (4.22) 
holds whenever 

Aq € V(v'lq') and v e V(v'lq')° 
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for some q' - {u', v') and w' - q') satisfying 

w' € R(q'), \\q' - g'll < t, \\w' — wll < t. 

With q' and w' fixed, Lemma 4.4 now shows the existence of a constant c > 0 such that 
(4.23) ||T*Aw - z||^ > c||Aw||^ 

for all 

(Aw,z) e (X X V{v'\q')) X (X X V{v'\q')f, 
with c depending only on ||X||, cg, and ck,v- Therefore (4.23) holds for all 

(Aw, z) € WjJ(g'|w). 

Applying (4.23) in the expression for a in (3.9) now shows that 

a{q\w;R) > c. 

Finally, an application of Lemma 3.4 shows that £i(-i{'w\q) < 00. 

In the other direction, to show that £j^-i{w\q) = 00 if b{q\'w;R) = 0 , we assume to the contrary 
that £j^-i{w\q) < 00. Then a{q\'w;R) > c for some constant c > 0 . Now we perform the above 
steps in the opposite direction to show that b{q\w;R) > 0, in contradiction to the premise. □ 

The following theorem, which specializes Lemma 4.6 to the specific structure assumed in 
this section and estimates the lower bounds slightly to derive easier conditions, is one of the 
main results of this work. 


Theorem 4.7. Let q,'weQ = W = XxY and let U be a neighborhood of{q, w). Suppose that 

Riq') - Piq') + h{q ) for P(q') = | ) 

for G and F* of the form (2.5) for some regular integrands g and f*, respectively. Assume further 
thath e C^(X; Y) and G e C^(X), and thatF* satisfies for some y > 0 the inclusion 


(4-24) 


D[dF*]{v'\q'){Av) D 


I yAn + VaFfiv'\q')°, Av € Vapfiv'lq'), 
| 0 , Av ^Vapf'v'lq'). 


In addition to these structural assumptions, suppose that there exists a constant cg > 0 such that 
(4.25) (V^G(u)? + f) > cgII^II^ e X). 

Define for 


and 


W. 


dP 


B := Vh{u){V'^G{u) + V„[V/i(u)*t;]) Vh{u) 


..{v\q) := {Vap-iv'\q') X Vapfivlff \ q' e dF*{v'), \\v' - n|| < t, \\q' - q\\ < t} 
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the quantity 


(4.26) b{q\'w;R) := sup inf 

t>o 

Then 

(4.27) “ 
provided 

(4.28) ma.x{Y,b(q\'w;R)} > 0. 


\\Bz-v\\ 


(z, v) € WQp,{v\q + h{u)), Zth 


If (4.24) holds as an equality, then (4.27) holds if and only if (4.28) holds. 
Proof Similarly to Proposition 4.2, we compute 


DR(q'\w'){Aq) 


/V^Giu'XAu) + Vu[Vh(u')*v']Au + Vh(u')*Av\ 

\ DldF*](v'jq'+ h(u'))(Av) - Vh(u')Au j’ 


and 

DR{q'\w')(Aq) = 
where we denote 


V^ Gju'XAu ) + Vu[Vh(u'yv’]Au + Vh{u')*Av 
D{dF*Xv'\q' + h(u'))(Av) - Vh(u')Au 


V„[V/i(u')*t'']Au := V (u I— > [Vh{u)*v']Au) {u). 

The structural assumptions of Lemma 4.6 are thus satisfied with 

Gq’ := V^G{u ) + Wu[^h{u')*v'], Ku' '■= Vh{u'), and V{q\\\!') := VdF*{p'\q' + h{u')). 


Moreover, Graph R fMJ is closed due to the assumptions on h and G and to G and F* being 
convex. Further, (4.6) holds by Corollary 2.7. 

Condition (4.18) is guaranteed by (4.25), while for (4.21), we first of all observe that 


B - KuGfKlAq, 


so (4.19) becomes 


(4-29) 


b{q\w,R) - sup inf 

^>0 ((0,A;7),(0,r))£W^(ij|'w), 


\\BAq - V 

IIA/7II 


Here _ 

- U X r) I C e W^iq\w)\ 

with 




Vdpf'v'lq' - h{u)) 


- VG(u') + Whiu'fv', 

q' e dF*iv') - h{u'), 


\W-q\\<t, I 

||w' - w|| < t J 
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We derive for small f > 0 and some constant C > 1 (depending on {q, w)) the inclusion 

||u' - m ||2 + \\v' - vf < f 2 , q' e dF*{v'), 

||VG(u') + Vhiu'fv' - ^||2 + \\q' - h{u') - 
\\u' - u ||2 + lit;' - t >||2 < q' e dF*{v'), 
\\q'-h{u')-qr<t^ 

\\u' - u\\ < t, \\v' - t;|| < t, q' € dF*{v'), 
\\q'-h{u)-q\\-\\h{u')-h{u)\\<t 

\\v' - t;|| <t,q'e dF*{v'), 1 ^ 

||;7'-h(u)-;7l| <a f 

In the final step we have used the fact that h e C^(X; Y) is Lipschitz on B(u, t) for small f > 0 . 
Now 

X r I lA € ^c} C W^^.{v\q + h{u)). 

Since we take the supremum over f > 0 in (4.19), the scaling factor C > 0 disappears, and we 
deduce from (4.26) and (4.29) that 

b{u\'w;R) > b{u\-w;R). 



^ ^VdF-i^'W) 
c ^VoF-(v'\q') 

C I^VaF-{v'\q') 
C ^VaF-'{v'\q') 


Thus (4.26) guarantees (4.19). Similarly, retracing the steps, we verify that 

Wgp,(v\ri-ku)) = \J{VxV° \V and c W^^\q\w) 

for some C2 > 1 . Indeed, using 

VG(u) + Vhiufv = 


we compute for some Ci > 1 that 


= ^VaF^iv'W) 
= ^VaF-{v'\q') 
C ^VaF*{v'\q') 


||t;' - t;|| < t, q' € dF*(v'), 1 
\\q'- fiiu) - q\\ < t J 

||f' - f II < t, q' e dF*(v'), 1 

II VG(u) + VhiuTv - ^11 + 11 , 7 ' - h{u) -q\\<t] 

||t;' - t;|| < t, q' e dF*(v'), 1 

||VG(u) + Vh(u)*v' - fll + ||;;' - h{u) - q\\ < Cit J 

||u' - u|| + ||t>' - t;|| < t, q' e dF*{v'), 

IIVG(u') + ^hiu'yv' - f||2 + 11;;' - h{u’) - qf < Czt' 




Hence, 

b{u\'w,R) > b{u\'w;R), 


and in particular, (4.19) guarantees (4.26). Our claims now follow from an application of Lemma 4.6, 
since its continuity requirements on G and K follow from the assumptions on G and h. □ 


In the remainder of this section, we apply Theorem 4.7 to show several stability properties of 
saddle points to (1.3). 
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4-3 METRIC REGULARITY OF THE LINEARIZED VARIATIONAL INCLUSION 

We begin with the simplest example of verifying ^Hy < oo with fixed u -u for q solving 

0 € ffj, ^(g)- This is useful for showing convergence of the primal-dual algorithm of [50]. By 
Proposition 4.1, we are in the setting of Theorem 4.7. Indeed, for R - Hy^a, we obtain an instance 
of (4.16) with 

h{u') = VK{u)u' + Cq. 

Furthermore, we also have 


+ V„[Vh(w)*r;]^ = ccl, 

so we may take cq = am (4.25). If y > 0, (4.28) is trivially satisfied. By Theorem 4.7, we therefore 
obtain 

Thus has the Aubin property at (0, ^ provided y > 0. We summarize these findings in the 

Y 

following proposition. 

Proposition 4.8. LetG be as in (4.2), K e C^(X; Y), and let F* satisfy (4.1) and (4.10). Suppose that 
q solves 0 e Hy ^{q) for some y > 0. Then w 1—> has the Aubin property at (0|g) if and 

only ify > 0 or b(q\0;Hyj2) > 0. 

If y = 0, we have to prove existence of a lower bound cj<: y > 0 through b. This is significantly 
more difficult. With u -u, we use (4.5) to compute 


h(u) = WK(u)u + Cq = K{u) and Wh(u) = WK(u). 

Consequently, (4.26) can be expressed in the setting of this proposition as 
(4-30) 

Hg]0: H,) = sup inf ( ^ ' 

t>0 [ l|z|| 


0 T ze Vdffv'lq'), V € Vspfv'fq'f, 
q' € dF*(v'), \\v' - till < t, 

\W-m\\<t 

We will return to the issue of verifying - or disproving - the lower bound on h with specific 
examples in Section 5. 

4.4 STABILITY WITH RESPECT TO DATA 

We now want to study the stability of the condition 0 € Fl:^f^ with respect to perturbation of 
the data y^. This of course only makes sense if we equate the base point u in Hq to the solution 
u. Therefore, we define for variations Ay in the data 


= a ^sup inf 


||VX(w)VX(ii)*z - 


f>0 


(z, v) € Wqp,{v\K(u)), Z + b! 


jAy{u, v) := P{u, v) + h^y{u, v) 
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with 


h^y{u,v) 


I VK(u)*v \ 
\Ay-K{u)j 


and P{u, v) 


jVG(u)\ 
\dF*(v)l ■ 


We remark that due to the linear dependence of the optimality conditions 0 e jAy{u, v) on Ay, 
the stability with respect to Ay can be seen as a form of tilt-stability [15,16, 31-34, 38, 43] for 
saddle-point systems. 

Observe now that 

jAyiq) = Ro(q) + , 


and in particular that Jo - Ro- Thus (3.2) with R = Rq and w = (0, Ay) yields 


inf II?-1?|| < 4 -i( 0 | 5 )||Ay|| whenever ||Ay|| < p. 

q-.OeJ^yiq) 0 

If K e C^(X; Y), by Proposition 4.2, we can compute DRo{q\w). In fact, with 

h{u) = K{u), 


we see that Ro is an instance of the class covered by Theorem 4.7. Its application directly yields 
the following proposition. 

Proposition 4.9. LetK e C^(X; Y) and suppose that F* satisfies (4.1) and (4.10). Denote byqAy a 
solution to the optimality conditions (1.2) for the problem 

cc 

min max — ||u||^ + {K(u) — Ay, i;) - F*(v). 

U V 2 ' 

Suppose that a solution q -qo exists, and there exists a constant cg > 0 such that 

( 4 . 31 ) all^ll^ + > cgII^II^ (f € X). 

IfY >0 orb(^0-,Ro) > 0, then for some p,J > 0 there exist solutions q Ay with 

||?-?Ayll < ^llAyll whenever ||Ay|| < p. 

Note that for b(q\ 0 ;Ro), we obtain from (4.26) exactly the same expression as for b(q\0;H:[i) 
in (4.30), i.e., 

(4-32) 

0 z€ Vdpfiv'lq'), V € Vgpfiv'lq'y, 
q' € dF*{v'), \\v' -1;|| < t, 

\\q'- Kiu)\\ < t 


b(q\ 0 ;Ro) = a ^ sup inf 


||VX(if)VX(5)* 


f>0 


4.5 STABILITY WITH RESPECT TO THE MOREAU-YOSIDA PARAMETER 

Finally, we study the stability of the regularized optimality condition 0 € Fl^ yilq) with respect 
to the Moreau-Yosida parameter y. With P as in the previous section, we now set 

Jy{u, v) := Py{u, v) + hy{u, v), 
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with 


hy{u,v) 


I VK(u)*v \ 

\YV-K(u)j 


and Py{u,v) 


I^Giu)\ 


Observe that Jo - Rq. Let q solve 0 e Ro{^- Now (3.1) applied to Jy at q and w € Jy(^ gives 
with w - 0 and q = q the estimate 


(4.33) inf ||p-?|| < ^r-i(w|5)||7j,(5)|| whenever ||w|| < p. 

p-oejyip) 

Since 0 € Ro(^, we deduce that Wy := ( 0 , pu) e Jy(^- This quickly leads to the following 
proposition. 

Proposition 4.10. Let K e C^(X; Y), and suppose F* satisfies (4.1) and (4.10). Denote by qy a 
solution to the optimality conditions {1.2) for the problem 

minmax —||m||^ + {K(u),v) - Fl(v). 

U V 2 ^ 

Suppose a solution q = qo exists, b(q\ 0 -,Ro) > 0, and (4.31) holds. Then for some p,L > 0 there exist 
solutions qy with 

(4.34) \\q- qyW < iy whenever 0 < y < p. 

Proof. We may assume that ||ri|| 0 , because otherwise qy = q. With Wy = ( 0 , yd), as above, 

we expand (4.33) into 

inf lip - ^1 < yfr-iCwy leys'll, whenever 0 < y < ||t;||“^p. 

p-.oejyip) ■'y 

In order to derive (4.34), we only need to show the existence of a finite constant ij-i{wy |^ < cxj 
and integrate ||S'|| into the constant. For this, we simply apply Theorem 4.7 to P = Jy with 
h{u') = K{u'), and observe that b(^Wy;Jy) - ^(^ 0 ;Po)- This follows from the fact that the 
expression (4.26) only depends on y through the base point q + h{u), which in this case is 
yv + (—yd + K(u)) = K(u). Observe that (4.31) is equally independent of y. We can thus bound 
tj^i{wy\q) from above uniformly in y e [0, p]. □ 


5 APPLICATION TO PARAMETER IDENTIFICATION PROBLEMS 

We now discuss the possibility of satisfying the assumptions of the preceding propositions in 
the context of the motivating parameter identification problems (1.4) and (1.5). Since this will 
depend on the specific structure of the parameter-to-observation mapping S, we consider as a 
concrete example the problem of recovering the potential term in an elliptic equation. 

Let Q c be an open bounded domain with a Lipschitz boundary dQ. For a given parameter 
u e {v e L°°(Q) : v > e} U c X := L^(Q), denote by S(u) := y € Ff^(Q) c L^(Q) Y the 
weak solution of 

(Vy, Vt;) + (uy, v) = (/, v) (v € H\Q)). 

This operator has the following useful properties [28]: 
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(ai) The operator S is uniformly bounded in 17 c X and completely continuous: If for u e U, 
the sequence {u„} c U satisfies Un ^ u inX, then 

S{un) —> S{u) in Y. 

(a2) S is twice Frechet differentiable. 

(A3) There exists a constant C > 0 such that 

\\yS{u)h\y <C\\h\\x {ueU,heX). 

(A4) There exists a constant C > 0 such that 

\\V^S(u){h,h)\y<C\\hf^ {ueU,heX). 

Furthermore, from the implicit function theorem, the directional Frechet derivative VS{u)h for 
given h e X can be computed as the solution w e H^{Q) to 

(5.1) {Vw,Vv) + {uw,v) = {-yh,v) (v € 

Similarly, the directional adjoint derivative VS{u)*h is given by yz, where z e H^{Q) solves 
(Vz, Vi;) + {uz, v) = {—h, v) {v e 

Similar expressions hold for V^S(u){hi, h2) and V{VS{u)*hi)h2. Hence, assumptions (A3-A4) hold 
for VS* and Vu(VS(u)*i;) for given v as well. 

Other operators satisfying the above assumptions are mappings from a Robin or diffusion 
coefficient to the solution of the corresponding elliptic partial differential equation [9]. 


5.1 H FITTING 

Let us first consider the fitting problem (1.4). We are in the setting of (4.i)-(4.2). More 
specifically now 

P*(^)= f for /*(z) ^ q_i4](z), 

Jq 

where we allow the integral to be possibly infinite if the integrand does not satisfy f* ov € L^{Q). 
We also have 


j: 


G{u)= I g{u{x))dx for ^(z) =-|z| 


as well as 


K{u) = S{u) - y^. 

Thus, the saddle-point conditions (4.4) for (1.4) are given by 

, au + VS{u)v 
0 e ' ^ ^ 


^'[-i,i](S) + S(u) - y^j ■ 
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Metric regularity We first address metric regularity of (Proposition 4.8) when y = 0 . Recall 

from Proposition 4.8 that in this case we need to show that 

(5-2) 


= sup inf 


^>0 


v|| 


(z, v) e - S(ii)), ztO 


= a ^sup inf 


||VS(M)VS(i?)*z-v|| 


f>0 


0 z € VdF*(v'\r]'), V e Vgfif'lrj'y, 
rj' e dF*(v'), ||i;' - n|| < t, 

\W - iy^ - S(w))|| < t 


> 0 . 


Let us try to force z(x) 7!^ 0 as much as possible in (5.2). From Corollary 2.12, we obtain that 
z € satisfies 

|r;'(x)| = 1 and r]'(x) -p 0, 

|r;'(x)| = 1 and t]'(x) - 0, 

|r;'(x)| < 1 and r]'(x) = 0, 

r;'(x)| = 1 and r}'(x) + 0, 
r;'(x)| = 1 and r}'(x) = 0, 
i;'(x)| < 1 and r]'{x) - 0. 

Therefore, z(x) 0 can only happen if r]\x) = 0 . If 

^ - S(u) i: 0 (x € Q), 

that is, if the data is reached almost nowhere, then the condition II/7 - r]'\\ < t gives for any 
f > 0 for small enough t > 0 the estimate 

£\{x € Q I r,'(x) = 0}) < £. 

In consequence, 

-C‘^({x € Q I z(x) t 0}) < e. 

With this, we deduce that 

I| 2 ||l 1 ( 0 ) < IIJ{z?tO}lll2(n)l|2||L2(Q) < Vf||2||L2(Q). 

We furthermore have that 

||VS(ii)*z|| < Cllzll^qo); 

this follows from the fact that VS(u) : (Q) —> C(Q) for any s' > d due to the regularity of 

do. and u (see, e.g., [19, Thm. 6.3]) and hence that VS{u)* : C(Q)* —> W^’*(Q) for s < d together 
with the embeddings L^(Q) C(Q)* and W^’*(Q) ^ L^(Q) for s > 1 (d = 2) or s > 6/5 {d = 3 ). 

An application of these estimates with v = 0 in (5.2) yields 

< a-iC||VS(ii)||V?. 


[{ 0 }, 

(5.3) z(x) e j - sign r;'(^)[0> “)> 

[m, 

while V € Vgf{v'\r]')° satisfies 

i lR, 

signi;'(x)[0,oo), 

{ 0 }, 
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Letting e \ 0 , we deduce that 


b(^0;Hu) = 0 . 

If, on the other hand, the data is reached on a set E of positive measure, i.e., ff = 0 on E, we 
may choose z freely on E. However, the lower hound 

||VS(u)*z|| > c||z|| (z e L^{E)), 

does not hold in general (take any orthonormal basis of L^{E), which converges weakly but not 
strongly to zero, and use the fact that VS(u) is a compact operator from L^(Q) to 1 ?{Q.) due to 
the Rellich-Kondrachev embedding theorem). Again, = 0 . 

Therefore by Proposition 4.8, there is no metric regularity without some sort of regularization. 
On the other hand, with Moreau-Yosida regularization, i.e., for y > 0 , we always have metric 
regularity of at {q, 0) by the same proposition. 

Data stability The situation is very similar for stability with respect to data (Proposition 4.9) 
when y = 0 . Comparing (4.30) and (4.32), we see that we have to study whether 

b(^ 0 \Ro) = b(q\ 0 ;H^) > 0 . 

Hence, we again cannot have data stability without Moreau-Yosida regularization. With regu¬ 
larization, i.e., for y > 0 , we still need to prove (4.31). Using the reverse triangle inequality, the 
boundedness of the dual variable v(x) € [- 1 , 1 ] due to the choice of F*, and assumption (A4), we 
have that 

«ll^ll^ + (V,[VS(iI)*fi]^, 0 > (« - C)||f 11^ > CGlIf 11^ 

for a sufficiently large and hence data stability. 

Stability with respect to y Since Proposition 4.10 holds under exactly the same conditions 
as Proposition 4.9, we deduce that there is no stability with respect to the Moreau-Yosida 
parameter at y = 0. This is to be expected, as any addition of regularization will, whenever 
r]{x) - 0 , immediately force v{x) = 0 . At a point y > 0 , the stability can be proved similarly to 
the arguments in Proposition 4.10. 


5.2 1“ FITTING 


Let us now consider the L“ fitting problem (1.5). We are again in the setting of (4.i)-(4.2), this 
time with 

f f*i^ix))dx for /*(z) = d|z|, 

Jq 

and G and K as in the previous subsection. Hence, the saddle-point conditions (4.4) are now 
given by 


( au + VS{u)v \ 

5 sign(r;) + S(u) - y'^j ' 
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Metric regularity Again, for metric regularity of ffj (Proposition 4.8) we need to show 


> 0 . 


Let us force again z(x) + 0 . From Corollary 

[{ 0 }, 

(5.5) z(x) € j sign r]'(x)[ 0 , 00), 

[iR, 

while V € Vffp*{v'\rj')° satisfies 

( IR, 

signt;'(x)(-oo,0], 

{ 0 }, 


2.15, we obtain that z e satisfies 

W(x)\ + < 5 , 

\t]'(x)\ - 5 and v'(x) - 0, 

\t]'(x)\ - 5 and v'(x) 7^ 0, 


\r]'(x)\ -p 8 and v'ix) — 0, 
\r]'(x)\ = 5 and v'(x) - 0, 
v'(x) + 0, 


If t;(x) = 0 a. e., and ess sup^gQ \^{x)\ < 5 holds - meaning the constraint 
(5.7) |S(u)(x) - y‘^(x)| < 5 


is almost never active - then we can proceed as in Section 5.1 to show for any c > 0 for small 
enough f > 0 the estimate 

£\{x € Q I \ri'ix)\ = d}) < c. 

In consequence, z e satisfies 

£\{x e Q I z(x) ^ 0}) < £‘^({x e Q | \r]'(x)\ = d}) < c, 


and we deduce following Section 5.1 that 

b(^0;H^) = b(^0;Ro) = 0. 

Therefore, by Proposition 4.8, we have no metric regularity if the constraint (5.7) is almost never 
active. (Any small change could force it to be active, and hence cause a large change in the dual 
variable.) However, also if the constraint (5.7) is active on an open set E, we may reason as in 
Section 5.1 to show instability. The only way to obtain stability is therefore with Moreau-Yosida 
regularization. 


Data stability Stability with respect to data (Proposition 4.9) again requires that 

b(q\ 0 ;Ro) ^ b{q\ 0 ;Hji) > 0. 

Hence, we cannot have data stability without Moreau-Yosida regularization. If y > 0 , we 
additionally need to prove (4.31). Using the reverse triangle inequality and assumption (A4), we 
have that 

«ll^ll^ + (v.[vs(S)*f7]^,0 > («-c)ii^ii^ > cgII^ii^ 

for a sufficiently large and hence data stability. Since in this case we do not have an a priori 
bound on v, the choice of a depends on v and hence on the data y^. 
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Stability with respect to y As in the case of L^fitting, stability with respect to the Moreau- 
Yosida parameter only holds at y > 0 . 


5.3 REGULARIZATION THROUGH PROJECTION 

Discretization provides an alternative to regularization. Indeed, in practice, the data lies in a 
finite-dimensional subspace Y' c Y - L^{Q). With !P the orthogonal projection from Y into Y', 
we then replace the fitting term Y by Fp := Yo!P. We then have withy = y'+y'‘~ € Y'0(Y')-‘- = Y 
that 

ve{{Y'yy = Y\ 


Fp{v) = sup (y-^, v) + (sup (y', v) - F{y')) = 
y' 


V i Y'. 


We emphasize that in this approach we only discretize the fitting term, while the nonlinear 
operator S and the regularizer G remain infinite-dimensional. 

Hence, 

\dF*{v) + (Y'y, veY', 

I0, v^Y'. 


dF*p{v) - 


From the definition (2.4) of the graphical derivative, we calculate that if either ^ Y' or At; ^ Y', 
then D[ 5 Y^](t;|;;)(At;) = 0 . When t; € Y' and At; € Y', we have for any ij € dF*(v) H (/; -H (Y')-*-) 
the inclusion 

D[dF*^](v\rj)(Av) = D[PdF*](v\r])(Azj) + (Y'^ 

D 'PD[dF*](v\fj)(Av) + (Y'y 
= D[dF*]{vmAv) + {Yy. 

Consequently, by basic properties of convex hulls, we also have the inclusion 


D[dF^]{v\r]){Av) D 


I D[< 9 Y*](t;|^)(At;) -t (Y')"", t;, At; eY',r]e dF*(v) + (Y'y, 

0, otherwise. 


Suppose now that DF* satisfies (4.24), that is. 


D[dF*](v'\r]')(Av) D 


I yAt; -E Av € VoF‘i'o’\T]'), 

I0, Av 


Since any cone V and subspace Y' satisfy the easily verified identity 

(V n Y')° = -E (Y')-", 


it follows for V, Av e Y' and rj e dF*{v) + (Y')-*- that 


(5.8) 


D[dF*^]iv'\t]')iAv) D 


iyAv + VaF^v'lfjr +{Y'y, 

I0, 


Av e V3F-{v'\fj)nY', 
otherwise. 


This holds for arbitrary fj e dF*{v) F{r] + (Y')-*-). In fact, in the following, we let fj e dF*{v) be 
the free parameter, and take, for example, rj - fj. 
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Let now ;p and be defined by (4.5) and (4.6), respectively, with Fp in place of F. An 
application of Theorem 4.7 to the inclusion (5.8) then gives the lower bound 


(5-9) 


b(^ 0 \Hu^p) = b{^ 0 ;Ro,p) 
( 


> a ^sup inf■ 
t>o 


||VS®VS®*z-v|| 


0 z e VdF*(v'\fi) n Y', 

veVaF<i^'\fir + iYT, 

fj € dF*(v'), v' e Y', 

\\v' - v\\ < t, \\P{fj - 1^11 < t 


= a ^sup inf 

t>o 


||!PVS(i 7 )VS( 5 )*z - Pv 


0 t z e VaF*{v'\fj) n Y', 

V € VaF*{v'\fj)°, 

fj € dF*{v'), v' € Y', 

\\v' - ull < t, \\P{fj-fTi)\\ < t 




Let {ei,..., e^} be an orthonormal basis for Y\ and for any v e Y, denote Vi := (cj, v). Then 
(5.9) forces 


(5.10) - Vi\'^ < and '^Ifji - rfif < . 

i=l 1=1 

Suppose there exist for each i = 1 ,..., AT closed sets A,-, Bj c IR satisfying for each fj and v' 
with fj € dF*{v') the conditions 

(5.11a) either (vj ^ A, and rjj € Bi) or (vj e A, and rjj ^ B,), 

(5.11b) fji ^ Bi and z € VaF>(v'\fj) DY' Zi = 0 , 

(5.11c) vj ^ Ai and v e VaF'-{v'\fj)° => v, = 0. 


Observe that the condition (5.11a) is a type of strict complementarity condition. We observe the 
following two situations. 

(i) If S'; ^ Ai, (5.10) for small enough t > 0 forces vj ^ A,, and through (5.11a), fji € Bi. Thus 
by (5.11c), Vi = 0. 

(ii) Likewise, if ^ Bi, (5.10) gives r]j ^ A,. Thus by (5.11b), Zj = 0 , and by (5.11a), Vi e A,. 

Thus the situations are mutually exclusive, and, by (5.11a), one of them has to occur. Importantly, 
therefore, v,- = 0 has to hold when we do not have the constraint Zi - 0 . Therefore, letting v,- 
vary freely whenever we have the constraint z; = 0, and z; vary freely whenever we have the 
constraint V; = 0, we obtain from (5.9) the lower estimate 


(5-12) 


b(q\0;Hgp) = b(q\0;Ro^p) 

^ _i. ^\\\PVSiu)VSid)*Pz\\ 
^ oc mt (-——- 


z e T', Zj = 0 if Vi e A,- 

z e rl. 
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Since VS(u) is invertible (as the inverse of a linear partial differential operator; cf (5.1)) and the 
orthogonal projection "P is self-adjoint, the restriction of 'PVS{u)VS{u)*'P to Y' is a self-adjoint 
positive definite operator on the finite-dimensional space Y' and therefore boundedly invertible. 
This implies the existence of a constant c > 0 such that 

||!PVS( 5 )VS( 5 )*Pz|| > c||z|| (z € Y'), 

which yields = b(^ 0 ;Ro^p) > a~^c > 0 and therefore metric regularity. 

It remains to verify the conditions (5.11). For this, let us further assume that the discretization 
is piecewise constant, that is e, = for some subdomains Q, c Q with Xfii JO; - Jo- 

For the example, (2.29) then gives 

Wi\ < 1 ^ nxQi ^ 0 - 

For t > 0 small enough, (5.3) and (5.4) therefore give 

\v[\ < 1 Vi = 0 and Z; € IR. 

This suggests to take A, = {-1,1} to satisfy (5.11c). Then, for (5.11a) to be satisfied, fhe condition 
(2.29) gives the only possibility of Bi = { 0 }. Further, for the strict complementarity within (5.11a) 
to hold, it is necessary to impose that the middle two cases in (5.3) and (5.4) do not occur. This 
strict complementary condition may also be stated as 

(5.13) (1 - \vi\)rii = 0 and 0 < (1 - l^il) -t \^i\. 

To verify (5.11b), suppose that fji ^ B,. It may happen that /)jq; behaves wildly. Nevertheless, 
by fji ^ Bi, there are points x e Q,- where necessarily fj{x) + 0 . Hence the condition (2.29) 
forces 0 v'{xj = sign/)(x). Thus z(x) = 0 , which through z €Y' forces z,- = 0 , proving (5.11b). 
Thus (5.11) and the estimate in (5.12) hold for projection-regularized fitting under the strict 
complementarity condition (5.13). 

For L°°-fitting, still using piecewise constant discretization, studying (2.33), (5.5), and (5.6), 
we see that we can take Ai = {0} and Bi = {-d, 5 } to obtain the same results under the strict 
complementarity condition 

(5.14) Vi {5 - \rii\) = 0 and 0 < |i;,-| -t (5 - \r]i\). 

Note that the strict complementarity conditions (5.13) and (5.14) can always be satisfied after 
a small perturbation of v, if necessary. Indeed, if fj'i is active {fji - 0 resp. \f]i\ = 5 ), then by 
(2.29) resp. (2.33), ffi can be made inactive - in which case the strict complementarity condition 
is satisfied - while maintaining the optimality condition fj e dFp(v). This change in v can, 
however, alter the constant in (4.31). 

Observe further that for the original infinite-dimensional problem, similar strict comple¬ 
mentarity conditions (pointwise almost everywhere) could be derived, but these would not 
be sufficient to obtain metric regularity since we still have the problem that the inverse of 
VS(m)VS(m)* is unbounded on L^(Q). Further, in the topology, even a strong complementarity 
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condition (with e lower bound in the inequality) would not be sufficient to transport it from the 
optimal solution to the perturbed variables v' and rj'. 

For stability with respect to perturbations of the data, condition (4.31) is also required. Since 
this condition is independent of F, it will hold under discretization whenever it holds for the 
original problem (with a possibly different constant cg since now v €Y' <z Y). 

6 CONCLUSION 

The purpose of this work was to derive explicit stability criteria for solutions to saddle-point 
problems in Hilbert spaces, in particular those arising from the minimization of nonsmooth 
nonlinear functionals commonly occurring in parameter identification, image processing, or 
PDF-constrained optimization problems. Our main results are a pointwise characterization of 
regular coderivatives of convex subdifferentials of integral functionals and explicit conditions 
for metric regularity of the corresponding variational inclusions. These make it possible to verify 
the Aubin property for concrete problems. While the results for our model problems are mostly 
negative (no regularity unless regularization or discretization is introduced), they are still useful: 
Our function-space analysis provides a unified framework for any conforming regularization; 
in particular, it shows that the stability properties are independent of the discretization of the 
unknown parameter. Furthermore, for arbitrary small fixed Moreau-Yosida parameters, the 
properties are also independent of the discretization of the data; this is especially important for 
the convergence of numerical algorithms, where this translates in a discretization-independent 
number of iterations required to reach a given tolerance. 

This work can be extended in a number of directions. In a follow-up paper, we will apply our 
results on the Aubin property of pointwise set-valued mappings to the convergence analysis 
of the nonlinear primal-dual extragradient method from [50] in function spaces. We also plan 
to investigate the possibility of obtaining partial stability results with respect to only the 
primal variable without regularization or discretization. An alternative would be to exploit the 
uniform stability with respect to regularization for fixed discretization, and with respect to 
discretization for fixed regularization, to obtain a combined convergence for a suitably chosen 
net (y, h) —> (0,0); this is related to the adaptive regularization and discretization of inverse 
problems [25]. Furthermore, it would be of interest to extend our analysis to include nonsmooth 
regularizers G, which were excluded in the current work for the sake of the presentation. It would 
also be worthwhile to try to adapt the stability analysis to make use of the limiting coderivative 
and its richer calculus; in particular to remove the geometric derivability assumption by directly 
working with the limiting coderivative. Finally, the pointwise characterization of coderivatives 
could be useful in deriving more explicit optimality conditions for bilevel optimization problems. 


APPENDIX A CODERIVATIVE FORMULA 

In this appendix, we give an explicit characterization of the regular coderivative for a class of 
set-valued mapping covering the examples in Section 2.4. 
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Proposition A.i. For a set-valued operator R : Q ^ Q, on a Hilbert space Q, suppose that 


(A.l) 


DR(q\w)(Aq) = 


h,Aq + V(q\wr, 

1 ®’ 


Aq € V{q\w), 
otherwise, 


for a linear operator T ■= Tq : Q ^ Q, dependent on q but not w, and a cone V{q\w) c Q, 
dependent on both q and w. Then 


(a.2) 


D*R(q\w){Aw) - 


I T*Aw + V(q\w)°, -Aw € Viq\w)° 


0 , 


otherwise. 


Note that if the cone V{q\w) is closed and convex - in particular, a closed subspace - then 
V{q\w)°° = V{q\w). 

Proof. We denote for short V := V{q\w). The expression (a.i) simply says that 

conv(T((q, w); Graph DR)) = {(Aq, Aw) e Q X Q j Aq e V, Aw € TqAq + y°}. 

We already know from Section 2.3 that 

N((q, w); Graph DR) - T({q, w); Graph DR)° = conv{T((q,w)-,GraphDR))°. 


Thus, 


{Aq',Aq) + (Aw', Aw) < 0 
for Aq eV,Awe TqAq + W 

(Aq' + T*Aw', Aq) + (Aw', Aw) < 0 


for Aq € y. Aw € W 


N((q, w)-. Graph DR) - |(A(3'', Aw') € 

- j(Ag',Aw')€02 

- ^(Aq, Aw') e Aq' + r*Aw' e y°. Aw' e F 

By Definition 2.3, we have 

D*i?(q'I w)(Aw) := jAq' € Q \ (Ag,-Aw) € ^(((j, w); Graph J?) [, 
from which (A. 2 ) follows. 
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